• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

How work the fog storage (nodes, groups, replication ...)

Scheduled Pinned Locked Moved
General
4
17
8.4k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • A
    arnaudrigole
    last edited by May 21, 2016, 3:58 PM

    Hello everybody,

    i use fog in production in my company since 6 months. I encountered many, many, many, many issues with it : kernels, nfs, ftp, tftp, pxe …
    BUT, some times it works, and its very helpful ! 🙂

    I have a question , in my setup, i’ve TWO storage nodes, one is fog local storage (fogserver/images)
    the other is located on a windows 2008r2 server, which work as a NFS server.

    I created two storage groups : LOCAL and EXTERNAL, respectively for local storage & windows storage
    When we create an image, we set it on a STORAGE GROUP and no on a storage node, WHY?

    Why do we use storage groups ?? i found nothing about “fog replication” but i’ve heard this “plugin” (?) is able to copy some images of a group to another group, but no procedure about it 😕

    Last question is, when i upload an image which is member of EXTERNAL group, the client will upload directly his .img files on windows server, OR he sned it on local fog server, then transfer it via FTP on windows server ? i don’t understand it…

    Thanks for your help

    Arnaud

    1 Reply Last reply Reply Quote 0
    • X
      x23piracy
      last edited by x23piracy May 22, 2016, 3:13 AM May 22, 2016, 9:12 AM

      Hi,

      Images will be replicated when enabled (within each image definition) to all nodes that are in the same storage group.
      There is something in the wiki that explains it: https://wiki.fogproject.org/wiki/index.php?title=Managing_FOG#Storage_Management

      alt text

      alt text

      Regards X23

      ║▌║█║▌│║▌║▌█

      1 Reply Last reply Reply Quote 0
      • G
        george1421 Moderator
        last edited by May 22, 2016, 12:31 PM

        There are a few things there.

        For your windows storage node you must have more than just nfs services installed. You also need to have the ftp services running and configured properly for full function.

        Images are captured to the master node. The master node will replicate its images and snapins to all storage nodes in its storage group. For example lets say your HQ is in London, and you have two remote locations New York and Madrid. All three locations are in the same storage group. When a new image is added at the London location, fog replication will move the images to the other two location. In this configuration the location plugin is typically used so you can allocate storage nodes to a location and then host computers to those same location. With the location plugin in place computers booting in New York location will image using the storage node in New York and not the Master Node in London. It is quite a brilliant setup.

        Images are replicated from the master node to the storage nodes using ftp. With the ftp that is used in FOG a max bandwidth can be defined so that fog replication doesn’t consume your entire network bandwidth for image movements. This is the requirement for ftp with your windows storage node.

        You have to remember that a storage node is a slave node (windows or linux) there is no gui and everything has to be done from the master node. You do get extra functionality if you use an official FOG storage node that is not available with a windows storage node, but windows will work at a minimum level.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

        A 1 Reply Last reply May 22, 2016, 4:48 PM Reply Quote 0
        • A
          arnaudrigole @george1421
          last edited by May 22, 2016, 4:48 PM

          @george1421
          Hello George,

          Thank you for your explanations. So, if i understand well, fog replication isn’t a plugin? its already installed with basic configuration with master and slaves nodes? And, in my case with my windows server just acting as a nfs storage node, there is no replication because its node defined as a real fog storage node, just an external storage so i don’t have all the features like fog replication?

          @x23piracy
          This is a good context that shows this picture. Any questions: where the queue size is defined? when i upload an image on a storage group, it’s stored first on master node defined, then automatically replicated on all the others storage nodes? (like my first question to george, is fog replication basically installed & configured with fog)

          Thanks you guys! Knowing of fog operations will help me to locate the problems much easier.

          X G 3 Replies Last reply May 22, 2016, 5:17 PM Reply Quote 0
          • X
            x23piracy @arnaudrigole
            last edited by x23piracy May 22, 2016, 11:25 AM May 22, 2016, 5:17 PM

            @arnaudrigole yes it’s there out of the box. the only thing needed is a group with at least to storage destinations (master & storage node) (storage node & storage node):

            alt text

            If you want separate storage location you have to create a storage group with at least one storage node or another master.

            alt text

            alt text

            alt text

            Regards X23

            ║▌║█║▌│║▌║▌█

            1 Reply Last reply Reply Quote 0
            • G
              george1421 Moderator @arnaudrigole
              last edited by May 22, 2016, 7:12 PM

              @arnaudrigole You are right fog replication isn’t a plugin it is built into the system. What IS a plugin that you will probably want is the location plugin. That allows you to assign storage nodes and client computers together by site. But storage nodes and master nodes are part of FOG 1.2.0 trunk build (maybe 1.2.0 stable but I haven’t used that in over 1.5 years).

              Just for clarity the target computers speak to the storage nodes (master or slave) over NFS. The master and storage nodes replicate with FTP.

              I started to put together a tutorial on setting up a 2012 server as a storage node. I ran into an issue and then the developers added some code to make it work. But, I haven’t had time to finish the document because of my work schedule. Here is the basics of what has to be setup to make a windows slave storage node. https://forums.fogproject.org/topic/6941/windows-server-as-fog-storage-node-reboot
              I understand you are using 2008 R2 server so some of the powershell commands won’t work, but you should see what needs to be done. Once you have FTP server installed and setup correctly on the windows storage node the images should replicate even if you don’t have the NFS part setup.

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

              A 1 Reply Last reply May 22, 2016, 9:47 PM Reply Quote 0
              • G
                george1421 Moderator @arnaudrigole
                last edited by May 22, 2016, 7:15 PM

                @arnaudrigole said:

                (like my first question to george, is fog replication basically installed & configured with fog)

                When FOG is setup replicated is installed and configured for the Master node only. When you add a FOG storage node then the slaves are configured as part of the setup process. When you add a Windows slave storage node then you have to setup the storage node configuration on the Master node by hand.

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                1 Reply Last reply Reply Quote 0
                • A
                  arnaudrigole @george1421
                  last edited by May 22, 2016, 9:47 PM

                  @george1421
                  George, could you explain how to do that powershell cmds in 2008r2? or maybe with graphic interface?

                  Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpServer.security.ssl.controlChannelPolicy -Value 0
                  Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpServer.security.ssl.dataChannelPolicy -Value 0
                  Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpServer.security.authentication.basicAuthentication.enabled -Value $true
                  Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpserver.userisolation.mode -Value 4
                  Add-WebConfiguration "/system.ftpServer/security/authorization" -value @{accessType="Allow";roles="fog_users";permissions="Read,Write";users=""} -PSPath IIS:\ -location "FOGFtpSite"
                  Restart-WebItem "IIS:\Sites\FOGFtpSite"
                  
                  
                  X 1 Reply Last reply May 22, 2016, 9:49 PM Reply Quote 0
                  • X
                    x23piracy @arnaudrigole
                    last edited by May 22, 2016, 9:49 PM

                    @arnaudrigole put the lines into a ps script .ps1 and create snapin (powershell based)

                    alt text

                    Regards X23

                    ║▌║█║▌│║▌║▌█

                    A 1 Reply Last reply May 22, 2016, 9:50 PM Reply Quote 0
                    • A
                      arnaudrigole @x23piracy
                      last edited by arnaudrigole May 22, 2016, 3:51 PM May 22, 2016, 9:50 PM

                      @x23piracy
                      I don’t understand? i have to setup the ftp server on windows 2008r2, which isn’t a client of fog
                      And again the powershell file will not work on 2008r2 which is radically different of 2012 😕

                      X 1 Reply Last reply May 22, 2016, 9:51 PM Reply Quote 0
                      • X
                        x23piracy @arnaudrigole
                        last edited by x23piracy May 22, 2016, 3:51 PM May 22, 2016, 9:51 PM

                        @arnaudrigole oh sorry i thought you would like to execute it after client imaging

                        ║▌║█║▌│║▌║▌█

                        A 1 Reply Last reply May 22, 2016, 9:56 PM Reply Quote 0
                        • A
                          arnaudrigole @x23piracy
                          last edited by May 22, 2016, 9:56 PM

                          @x23piracy
                          I’m just searching how to make my setup work …
                          Need to store images on that f****g 2008r2 and its really HARD! Currently, my problem is located on image upload: when the upload task finish, the .Img files stay on /images/dev/mac@ddrs/ and they don’t move in /images/imagenamedefinedonwebgui

                          Some people of fog support told me that its an ftp issue, (FTP to transfer /images/dev on /images , why don’t use a simply CP cmd, in local situation???) and i don’t understand why. It will be never resolved, because i think nobody understand (my fault maybe, i don’t write english very well) my configuration and issues i encounter :(((! I needed to write over 7 or 8 threads for the same issue, each time we resolve 1 step, but each next step i encounter another issue !!..

                          X G 2 Replies Last reply May 22, 2016, 10:19 PM Reply Quote 0
                          • X
                            x23piracy @arnaudrigole
                            last edited by May 22, 2016, 10:19 PM

                            @arnaudrigole look over here please: https://forums.fogproject.org/topic/1811/how-to-setup-a-fog-storage-node-without-installing-fog

                            ║▌║█║▌│║▌║▌█

                            1 Reply Last reply Reply Quote 0
                            • G
                              george1421 Moderator
                              last edited by May 23, 2016, 1:52 AM

                              I translated the 2012 instruction to Windows 2008 commands. I have not tested these commands yet, but in general they should configure a windows 2008 server into a FOG storage node.

                              https://forums.fogproject.org/topic/6941/windows-server-as-fog-storage-node-reboot/9

                              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                              1 Reply Last reply Reply Quote 0
                              • G
                                george1421 Moderator @arnaudrigole
                                last edited by george1421 May 22, 2016, 8:02 PM May 23, 2016, 2:01 AM

                                @arnaudrigole said :

                                Some people of fog support told me that its an ftp issue, (FTP to transfer /images/dev on /images , why don’t use a simply CP cmd, in local situation???)

                                You have to remember the brains for FOG is on the FOG Master node. Consider if your storage node is located in another city connected via MPLS. A copy command issued by the FOG master node, would transfer the image from the storage node /images/dev/macaddr to the master node and then from the master node to /images/{fog_sys_name}. Now with FOG using ftp, the FOG master node connectes to the storagenode via FTP and then issues the move command via ftp. The FTP Server (in this case the storage node) just moves the file locally and no network traffic is generated. Once the move is complete the FOG server verifies the move and then disconnects.

                                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                A 1 Reply Last reply May 23, 2016, 6:53 AM Reply Quote 0
                                • A
                                  arnaudrigole @george1421
                                  last edited by May 23, 2016, 6:53 AM

                                  @george1421
                                  Oh ok, thanks for the precisions.Let’s work about ftp issues!
                                  Thank you for your work george, appreciate it !!!
                                  I’ll try it and keep you up.

                                  1 Reply Last reply Reply Quote 0
                                  • W
                                    Wayne Workman
                                    last edited by May 23, 2016, 2:55 PM

                                    This might help:
                                    https://wiki.fogproject.org/wiki/index.php?title=Location_Plugin

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                    Daily Clean Installation Results:
                                    https://fogtesting.fogproject.us/
                                    FOG Reporting:
                                    https://fog-external-reporting-results.fogproject.us/

                                    1 Reply Last reply Reply Quote 0
                                    • 1 / 1
                                    1 / 1
                                    • First post
                                      7/17
                                      Last post

                                    139

                                    Online

                                    12.0k

                                    Users

                                    17.3k

                                    Topics

                                    155.2k

                                    Posts
                                    Copyright © 2012-2024 FOG Project