• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Storage nodes not deploying images

    Scheduled Pinned Locked Moved Solved
    FOG Problems
    3
    31
    6.9k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Wayne WorkmanW
      Wayne Workman @EuroEnglish
      last edited by

      @EuroEnglish Going to look through those closely, but this is what I was requesting for the image’s storage group settings:
      0_1491953136260_Image Storage Group settings.png

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
      Daily Clean Installation Results:
      https://fogtesting.fogproject.us/
      FOG Reporting:
      https://fog-external-reporting-results.fogproject.us/

      E 1 Reply Last reply Reply Quote 0
      • Wayne WorkmanW
        Wayne Workman @EuroEnglish
        last edited by

        @EuroEnglish Can you also please post a screenshot of your location management area - the one that lists all the locations please? Also - ensure as @george1421 already said - make sure the host you’re trying to image is assigned the location you want it to use.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
        Daily Clean Installation Results:
        https://fogtesting.fogproject.us/
        FOG Reporting:
        https://fog-external-reporting-results.fogproject.us/

        E 1 Reply Last reply Reply Quote 0
        • george1421G
          george1421 Moderator @EuroEnglish
          last edited by george1421

          @EuroEnglish said in Storage nodes not deploying images:

          me to push around 20 at a time without too much slow down (

          If you are going to do this, and all clients and fog server will be on the same subnet (VLAN) you might consider a multicast deployment instead of 20 unicast deployments. You can scale to 50 or more systems using a mutlicast deployment. It does take some work to set it up but you can deploy a whole classroom quickly with that route.

          <edit> Also consider if your network is robust enough to support 20 unicast streams running at 4-6GB/min. That may be too much for your infrastructure. </edit>

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

          E 2 Replies Last reply Reply Quote 2
          • E
            EuroEnglish @Wayne Workman
            last edited by

            @Wayne-Workman

            Hi Wayne,
            Sorry about that, I have now taken a screenshot of that section and attached it. I also checked the other 2 images I have and all are the same as the one attached.

            0_1491953436723_Fog Image Management 2.jpg

            1 Reply Last reply Reply Quote 0
            • george1421G
              george1421 Moderator
              last edited by

              looking at the log, it might appear that your storage groups are not setup correctly.

              [04-11-17 4:06:27 pm] | Image Name: Student-E460
              [04-11-17 4:06:27 pm] * Found Image to transfer to 2 nodes
              [04-11-17 4:06:27 pm] * Attempting to perform Group -> Nodes image replication.
              [04-11-17 4:06:27 pm] | There are no other members to sync to.
              [04-11-17 4:06:27 pm] | Image Name: Student-Twist
              [04-11-17 4:06:27 pm] * Not syncing Image between groups
              [04-11-17 4:06:27 pm] | There are no other members to sync to.
              [04-11-17 4:06:27 pm] | Image Name: Student-T440-Old
              [04-11-17 4:06:27 pm] * Not syncing Image between groups
              [04-11-17 4:06:27 pm] | There are no other members to sync to.
              [04-11-17 4:06:27 pm] | Image Name: Student-E460
              [04-11-17 4:06:27 pm] * Not syncing Image between groups
              

              This part of the log may need some dev clarification (and some possible code changes). There is no need to sync because??? The file already exists on the target server or the FOG replicator is being a bit lazy??

              [04-11-17 4:06:30 pm] | Student-E460: No need to sync d1.partitions file to FOG-Ubuntu-Slave1
              [04-11-17 4:06:30 pm] | Student-E460: No need to sync d1.original.swapuuids file to FOG-Ubuntu-Slave1
              [04-11-17 4:06:30 pm] | Student-E460: No need to sync d1.original.fstypes file to FOG-Ubuntu-Slave1
              [04-11-17 4:06:29 pm] | Student-E460: No need to sync d1.minimum.partitions file to FOG-Ubuntu-Slave1
              [04-11-17 4:06:29 pm] | Student-E460: No need to sync d1.mbr file to FOG-Ubuntu-Slave1
              [04-11-17 4:06:29 pm] | Student-E460: No need to sync d1.fixed_size_partitions file to FOG-Ubuntu-Slave1
              

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

              Wayne WorkmanW E 2 Replies Last reply Reply Quote 1
              • E
                EuroEnglish @Wayne Workman
                last edited by

                @Wayne-Workman

                Hi Wayne,
                Here is the location management screenshot.

                0_1491953785511_Fog Location management.jpg

                george1421G 1 Reply Last reply Reply Quote 0
                • george1421G
                  george1421 Moderator @EuroEnglish
                  last edited by

                  @EuroEnglish Sorry to keep asking you for screen shots and more screen shots. The one I see missing is the storage group association. All the other ones look good.

                  Just be aware that the storage nodes and location plugins were intending for a multi site or multi building(subnet) setup where you would dedicate certain workstation to specific locations. Its not really setup to do what I’m calling overflow imaging. (you deploy to server 1 until max clients is reached, then the next client is assigned to another storage node until its max client level is reached and so on). The storage node concept doesn’t work that way.

                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                  george1421G 1 Reply Last reply Reply Quote 0
                  • E
                    EuroEnglish @george1421
                    last edited by

                    @george1421

                    Thanks for the idea regarding multicasting, I will look into setting that up in the Wiki - Seems almost everything you would ever need about Fog is there somewhere. We were running WDS on a Windows server for the last few years, but something went wrong and it will not capture new images. Still deploys, but not a good idea to have old images that take longer to update than it did to deploy the image 😉

                    I was researching how to fix the WDS server and came across a forum for Fog, this solution is a life-saver, no more sysprep and capturing images is much easier. I also find that deployment is much faster with Fog. We image the machines over the summer break, working in a high school, and bandwidth isn’t much of a problem, individual 1Gb Ethernet to each server and fiber point to point between the server room switches and my imaging location switches. However, I agree that multicasting might still speed things up.

                    1 Reply Last reply Reply Quote 0
                    • george1421G
                      george1421 Moderator @george1421
                      last edited by

                      @george1421 Something else to consider is that your image push time will be less than 5 min per system per unicast. Can you stage these systems (next system) that fast? It will take longer for windows to install than it will to push out 5 systems in series.

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                      1 Reply Last reply Reply Quote 0
                      • Wayne WorkmanW
                        Wayne Workman @george1421
                        last edited by Wayne Workman

                        @george1421 He has another group called default that doesn’t have any nodes in it. I know from the screenshot he posted for the Images Storage Groups. default is listed there. I’m guessing that’s what the logs are referring to.

                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                        Daily Clean Installation Results:
                        https://fogtesting.fogproject.us/
                        FOG Reporting:
                        https://fog-external-reporting-results.fogproject.us/

                        E 1 Reply Last reply Reply Quote 0
                        • E
                          EuroEnglish @george1421
                          last edited by

                          @george1421

                          Based on your thoughts about the Image replication log files I used Webmin to check the Images folder on each server. It seems that you might be correct, it shows that there is a small file size difference between the main server and the two storage servers. It also seems that rights didn’t transfer the same, both storage servers have different rights to the files.

                          I marked each file manager with the server for reference:

                          0_1491955601541_Fog File manager.jpg

                          george1421G 1 Reply Last reply Reply Quote 0
                          • george1421G
                            george1421 Moderator @EuroEnglish
                            last edited by george1421

                            @EuroEnglish The other thing I see is file ownership is different on each system too. That’s not right.

                            Fog will use the user account listed for each storage node to copy the files from the master node to the remote nodes. This is done over ftp from the master server in the storage group to all slave nodes in the same storage group. Systems defined in FOG but in a different storage group will be left out of the replication cycle.

                            Did you seed the files on each storage node or did FOG do this?

                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                            E 1 Reply Last reply Reply Quote 1
                            • E
                              EuroEnglish @george1421
                              last edited by

                              @george1421

                              Well spotted on the owner difference, I didn’t even notice that one - Strange as it pretty obvious now you mentioned it. All the image replication was done by Fog itself, however it could be that I messed something up when I was setting up the different users on the different servers - Again, could be part of the problem.

                              Wayne WorkmanW 1 Reply Last reply Reply Quote 0
                              • E
                                EuroEnglish @Wayne Workman
                                last edited by

                                @Wayne-Workman

                                I have deleted the ‘Default’ storage group, it will never be used and so I guess there is no point leaving it there. I will add another group, ‘Teacher’, to separate the different images, but that can wait until I have everything running correctly. That should clear up those log file entries I would imagine.

                                1 Reply Last reply Reply Quote 0
                                • Wayne WorkmanW
                                  Wayne Workman @EuroEnglish
                                  last edited by

                                  @EuroEnglish The fog installer along with a couple extra commands can fix these things.

                                  SSH into each box, become root with sudo -i
                                  Make sure that inside /opt/fog/.fogsettings (as root) the username field is set to fog. Then re-run the fog installer (again, as root). This will fix the passwords & user names both on the local system & in the DB. Then on each box, reset ownership & permissions of the images with chown -R fog:root /images;chmod -R 777 /images

                                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                  Daily Clean Installation Results:
                                  https://fogtesting.fogproject.us/
                                  FOG Reporting:
                                  https://fog-external-reporting-results.fogproject.us/

                                  E 1 Reply Last reply Reply Quote 0
                                  • E
                                    EuroEnglish @Wayne Workman
                                    last edited by

                                    @Wayne-Workman

                                    Hi Wayne,
                                    After a few small issues sorting out user names and passwords, I reran the Fog Installer and then set the ownership/permissions. That all looks good now. Thanks for that help.

                                    I ran into a problem with my second storage node and deleted it from Storage Management, then added it back in, which put it at the top of the list in “All storage nodes”. Now when I deploy to the 6 computers all of them deploy from this storage node, previously all pulled from my main server (which was at the top of the storage management list before).

                                    I think that my problem is that I am misunderstanding the ‘Max Clients’ option, I presumed that it would send an image to the host from one node, until it hits the max client number, then start sending from the next node automatically, and so on. I am not registering my hosts and simply selecting deploy, therefore it seems to be ignoring settings as the hosts aren’t running as clients. Could that be right?

                                    Thanks again

                                    Peter

                                    Wayne WorkmanW 1 Reply Last reply Reply Quote 0
                                    • E
                                      EuroEnglish @george1421
                                      last edited by

                                      @george1421

                                      Hi George,
                                      I just created a simple multicast session for 6 hosts as a test, it ran really well and seems faster than 6 individual unicast sessions. I am wondering how Fog processes multicast sessions, when the multicast was running I monitored the Fog Management home page and noticed that my main server was the only thing showing transmit data, however both my 2 storage nodes seemed to be receiving the same amount of data. Does multicast push from the main server to the hosts? Or, does the main server push to the storage nodes and they distribute the session? My bandwidth was fluctuating between 500 Mbps and 3000 Mbps, which tells me that all three servers must be pooling the multicast session, but maybe the bandwidth monitor on the dashboard isn’t accurate?

                                      Thanks again for all your help with this.

                                      Peter

                                      george1421G 1 Reply Last reply Reply Quote 0
                                      • Wayne WorkmanW
                                        Wayne Workman @EuroEnglish
                                        last edited by

                                        @EuroEnglish said in Storage nodes not deploying images:

                                        I am not registering my hosts and simply selecting deploy, therefore it seems to be ignoring settings as the hosts aren’t running as clients. Could that be right?

                                        This is why random storage nodes are being selected for imaging despite location settings - The location plugin only works with registered hosts. FOG for all intensive purposes doesn’t really care a lot about non-registered hosts. All of FOG’s features come to life when hosts are registered.

                                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                        Daily Clean Installation Results:
                                        https://fogtesting.fogproject.us/
                                        FOG Reporting:
                                        https://fog-external-reporting-results.fogproject.us/

                                        E 1 Reply Last reply Reply Quote 1
                                        • E
                                          EuroEnglish @Wayne Workman
                                          last edited by

                                          @Wayne-Workman

                                          Hi Wayne,
                                          I just created a simple multicast session, a great suggestion from George, for 6 hosts as a test, it ran really well and seems much faster than 6 individual unicast sessions. I am wondering how Fog processes multicast sessions, when the multicast was running I monitored the Fog Management home page and noticed that my main server was the only thing showing transmit data, however both my 2 storage nodes seemed to be receiving the same amount of data. Does multicast push from the main server to the hosts? Or, does the main server push to the storage nodes and they distribute the session? My bandwidth was fluctuating between 500 Mbps and 3000 Mbps, which tells me that all three servers must be pooling the multicast session, but maybe the bandwidth monitor on the dashboard isn’t accurate?

                                          I am trying to work out, once I start deploying 20 or more hosts at a time, what the best method would be. If multicasting utilizes the normal server and all storage nodes I can create a larger number of storage nodes to increase the combined available bandwidth. If multicasting only uses the normal server though, I may as well not have more than one storage node (a good way to make sure I have backups of images in case of normal server failure).

                                          I am not finding any information online about how the multicasting distributes workload and this could define whether smaller unicast deploys or larger multicast would work better for me.

                                          Thanks again for your help, really helped on the issues the Wiki didn’t cover.

                                          Regards

                                          Peter

                                          Thanks again for all your help with this.

                                          Peter

                                          george1421G 1 Reply Last reply Reply Quote 0
                                          • george1421G
                                            george1421 Moderator @EuroEnglish
                                            last edited by

                                            @EuroEnglish AFAIK, multicast sessions are only hosted by the master server. Unless something has changed in the multicast bits only the master server sends out the image.

                                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                            Wayne WorkmanW 1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 1 / 2
                                            • First post
                                              Last post

                                            240

                                            Online

                                            12.0k

                                            Users

                                            17.3k

                                            Topics

                                            155.2k

                                            Posts
                                            Copyright © 2012-2024 FOG Project