• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

Storage nodes not deploying images

Scheduled Pinned Locked Moved Solved
FOG Problems
3
31
6.9k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • E
    EuroEnglish
    last edited by Apr 11, 2017, 9:48 PM

    Server
    • FOG Version: 1.4.0-RC-4
    • OS: Ubuntu 16.04 LTS
    Description

    I installed Fog 1.3.5 on my normal server, added a couple of images and then deployed them without problems. This all went incredibly easily. I then created a new server and added a storage node, sorted out getting the images to replicate, but was unable to deploy from this new storage node - Everything still deployed from the main (normal) server. I made certain that max clients was set to 3 on each, yet when I deployed 6 machines at the same time all stayed on the main server. I was frustrated, but after a while found that the new storage node had installed with version 1.4.0, where the main server was running 1.3.5. I upgraded the main server and rebooted them both, happily I found it now push 3 deployments to each server (although the ones deploying from the storage node were quite a bit slower than those deploying on the main server).

    I then built another server, added fog and configured it as a storage node, set it up on for management site and the images copied over. I changed all the storage nodes to ‘max clients: 2’ and rebooted all three servers. It has now reverted to all 6 deployments pushing from the main server and nothing from the storage nodes.

    All three servers are running on Windows Hyper-V, all are running Ubuntu 16.04 LTS and all running Fog 1.4.0. Only the main server is set to ‘Master node’ right now.

    Any ideas why my two storage node servers aren’t picking up deployments and why the main server is accepting 6 clients when it is set to ‘Max clients: 2’?

    Thanks in advance

    Peter

    G 1 Reply Last reply Apr 11, 2017, 10:05 PM Reply Quote 0
    • G
      george1421 Moderator @EuroEnglish
      last edited by Apr 11, 2017, 10:05 PM

      @EuroEnglish Are you using the location plugin? You will use that to tell which clients to use what storage node (including the main fog server too).

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

      E 2 Replies Last reply Apr 11, 2017, 10:39 PM Reply Quote 2
      • E
        EuroEnglish
        last edited by Apr 11, 2017, 10:37 PM

        This post is deleted!
        1 Reply Last reply Reply Quote 0
        • E
          EuroEnglish @george1421
          last edited by Apr 11, 2017, 10:39 PM

          @george1421

          Hi George,
          I just watched the video tutorial on the location plugin, then installed it on the main server. I have created both the main server and two storage nodes in the location management tab. As all three are Fog servers, one normal and two storage, I selected the option to use kernels and inits from all locations. Time to test, I will post up once I have restarted my 6 test hosts and seen where they pull their deployment images from now.

          Thanks for the information, I didn’t know about the plugins as I only started Fog (to replace my WDS server) less than a week ago.

          1 Reply Last reply Reply Quote 1
          • E
            EuroEnglish @george1421
            last edited by Apr 11, 2017, 10:53 PM

            @george1421

            Hi George,
            It seems that the hosts are still deploying from the main server only, I attached a screenshot showing the bandwidth and it shows that only FOG-Ubuntu (my normal server) is sending data, my 2 slave (storage servers) nodes are doing nothing.

            I am also watching ‘tcptrack’ on all three servers, there is a lot of activity between the 2 storage nodes and the main server, but all very small and very short connections (speed less than 5 KB/s), nothing like the traffic on the main server - I see all the small activities, but also the 6 deployments running around 20 MB/s.

            I am wondering if this is because I am only selecting deploy at the Fog PXE window on the hosts and not registering the clients when they first connect?

            0_1491950726774_Fog deploy.jpg

            W G 2 Replies Last reply Apr 11, 2017, 11:06 PM Reply Quote 0
            • W
              Wayne Workman @EuroEnglish
              last edited by Apr 11, 2017, 11:06 PM

              @EuroEnglish Double check the image replicated properly. If the non-masters don’t have a copy, I think it’ll use the node that does have a copy.

              Can you post a screenshot of your Storage Management page please? the one that lists all the storage nodes, groups, master status. Also - the image you’re deploying, can you post a screenshot of the Image’s storage group area please?

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
              Daily Clean Installation Results:
              https://fogtesting.fogproject.us/
              FOG Reporting:
              https://fog-external-reporting-results.fogproject.us/

              E 1 Reply Last reply Apr 11, 2017, 11:22 PM Reply Quote 0
              • G
                george1421 Moderator @EuroEnglish
                last edited by Apr 11, 2017, 11:10 PM

                @EuroEnglish And also did you assign specific hosts to specific locations?

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                1 Reply Last reply Reply Quote 1
                • E
                  EuroEnglish @Wayne Workman
                  last edited by Apr 11, 2017, 11:22 PM

                  @Wayne-Workman

                  Hi Wayne,
                  I have attached the screenshots you requested and also the most recent Image Replication log file, in case that shows any issues. Right now it isn’t a problem, as I am only testing a few machines at a time, however once summer arrives I need to deploy around 300 laptops. I am hoping that setting up a main server and 3 storage nodes will allow me to push around 20 at a time without too much slow down (5 per node).

                  0_1491952887589_Fog Storage Management.jpg

                  0_1491952923745_Fog Image Management.jpg

                  0_1491952936259_Fog Image Replicator Log.txt

                  W G 3 Replies Last reply Apr 11, 2017, 11:25 PM Reply Quote 0
                  • W
                    Wayne Workman @EuroEnglish
                    last edited by Apr 11, 2017, 11:25 PM

                    @EuroEnglish Going to look through those closely, but this is what I was requesting for the image’s storage group settings:
                    0_1491953136260_Image Storage Group settings.png

                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                    Daily Clean Installation Results:
                    https://fogtesting.fogproject.us/
                    FOG Reporting:
                    https://fog-external-reporting-results.fogproject.us/

                    E 1 Reply Last reply Apr 11, 2017, 11:30 PM Reply Quote 0
                    • W
                      Wayne Workman @EuroEnglish
                      last edited by Apr 11, 2017, 11:28 PM

                      @EuroEnglish Can you also please post a screenshot of your location management area - the one that lists all the locations please? Also - ensure as @george1421 already said - make sure the host you’re trying to image is assigned the location you want it to use.

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                      Daily Clean Installation Results:
                      https://fogtesting.fogproject.us/
                      FOG Reporting:
                      https://fog-external-reporting-results.fogproject.us/

                      E 1 Reply Last reply Apr 11, 2017, 11:36 PM Reply Quote 0
                      • G
                        george1421 Moderator @EuroEnglish
                        last edited by george1421 Apr 11, 2017, 5:37 PM Apr 11, 2017, 11:29 PM

                        @EuroEnglish said in Storage nodes not deploying images:

                        me to push around 20 at a time without too much slow down (

                        If you are going to do this, and all clients and fog server will be on the same subnet (VLAN) you might consider a multicast deployment instead of 20 unicast deployments. You can scale to 50 or more systems using a mutlicast deployment. It does take some work to set it up but you can deploy a whole classroom quickly with that route.

                        <edit> Also consider if your network is robust enough to support 20 unicast streams running at 4-6GB/min. That may be too much for your infrastructure. </edit>

                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                        E 2 Replies Last reply Apr 11, 2017, 11:45 PM Reply Quote 2
                        • E
                          EuroEnglish @Wayne Workman
                          last edited by Apr 11, 2017, 11:30 PM

                          @Wayne-Workman

                          Hi Wayne,
                          Sorry about that, I have now taken a screenshot of that section and attached it. I also checked the other 2 images I have and all are the same as the one attached.

                          0_1491953436723_Fog Image Management 2.jpg

                          1 Reply Last reply Reply Quote 0
                          • G
                            george1421 Moderator
                            last edited by Apr 11, 2017, 11:35 PM

                            looking at the log, it might appear that your storage groups are not setup correctly.

                            [04-11-17 4:06:27 pm] | Image Name: Student-E460
                            [04-11-17 4:06:27 pm] * Found Image to transfer to 2 nodes
                            [04-11-17 4:06:27 pm] * Attempting to perform Group -> Nodes image replication.
                            [04-11-17 4:06:27 pm] | There are no other members to sync to.
                            [04-11-17 4:06:27 pm] | Image Name: Student-Twist
                            [04-11-17 4:06:27 pm] * Not syncing Image between groups
                            [04-11-17 4:06:27 pm] | There are no other members to sync to.
                            [04-11-17 4:06:27 pm] | Image Name: Student-T440-Old
                            [04-11-17 4:06:27 pm] * Not syncing Image between groups
                            [04-11-17 4:06:27 pm] | There are no other members to sync to.
                            [04-11-17 4:06:27 pm] | Image Name: Student-E460
                            [04-11-17 4:06:27 pm] * Not syncing Image between groups
                            

                            This part of the log may need some dev clarification (and some possible code changes). There is no need to sync because??? The file already exists on the target server or the FOG replicator is being a bit lazy??

                            [04-11-17 4:06:30 pm] | Student-E460: No need to sync d1.partitions file to FOG-Ubuntu-Slave1
                            [04-11-17 4:06:30 pm] | Student-E460: No need to sync d1.original.swapuuids file to FOG-Ubuntu-Slave1
                            [04-11-17 4:06:30 pm] | Student-E460: No need to sync d1.original.fstypes file to FOG-Ubuntu-Slave1
                            [04-11-17 4:06:29 pm] | Student-E460: No need to sync d1.minimum.partitions file to FOG-Ubuntu-Slave1
                            [04-11-17 4:06:29 pm] | Student-E460: No need to sync d1.mbr file to FOG-Ubuntu-Slave1
                            [04-11-17 4:06:29 pm] | Student-E460: No need to sync d1.fixed_size_partitions file to FOG-Ubuntu-Slave1
                            

                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                            W E 2 Replies Last reply Apr 12, 2017, 12:04 AM Reply Quote 1
                            • E
                              EuroEnglish @Wayne Workman
                              last edited by Apr 11, 2017, 11:36 PM

                              @Wayne-Workman

                              Hi Wayne,
                              Here is the location management screenshot.

                              0_1491953785511_Fog Location management.jpg

                              G 1 Reply Last reply Apr 11, 2017, 11:41 PM Reply Quote 0
                              • G
                                george1421 Moderator @EuroEnglish
                                last edited by Apr 11, 2017, 11:41 PM

                                @EuroEnglish Sorry to keep asking you for screen shots and more screen shots. The one I see missing is the storage group association. All the other ones look good.

                                Just be aware that the storage nodes and location plugins were intending for a multi site or multi building(subnet) setup where you would dedicate certain workstation to specific locations. Its not really setup to do what I’m calling overflow imaging. (you deploy to server 1 until max clients is reached, then the next client is assigned to another storage node until its max client level is reached and so on). The storage node concept doesn’t work that way.

                                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                G 1 Reply Last reply Apr 11, 2017, 11:46 PM Reply Quote 0
                                • E
                                  EuroEnglish @george1421
                                  last edited by Apr 11, 2017, 11:45 PM

                                  @george1421

                                  Thanks for the idea regarding multicasting, I will look into setting that up in the Wiki - Seems almost everything you would ever need about Fog is there somewhere. We were running WDS on a Windows server for the last few years, but something went wrong and it will not capture new images. Still deploys, but not a good idea to have old images that take longer to update than it did to deploy the image 😉

                                  I was researching how to fix the WDS server and came across a forum for Fog, this solution is a life-saver, no more sysprep and capturing images is much easier. I also find that deployment is much faster with Fog. We image the machines over the summer break, working in a high school, and bandwidth isn’t much of a problem, individual 1Gb Ethernet to each server and fiber point to point between the server room switches and my imaging location switches. However, I agree that multicasting might still speed things up.

                                  1 Reply Last reply Reply Quote 0
                                  • G
                                    george1421 Moderator @george1421
                                    last edited by Apr 11, 2017, 11:46 PM

                                    @george1421 Something else to consider is that your image push time will be less than 5 min per system per unicast. Can you stage these systems (next system) that fast? It will take longer for windows to install than it will to push out 5 systems in series.

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                    1 Reply Last reply Reply Quote 0
                                    • W
                                      Wayne Workman @george1421
                                      last edited by Wayne Workman Apr 11, 2017, 6:05 PM Apr 12, 2017, 12:04 AM

                                      @george1421 He has another group called default that doesn’t have any nodes in it. I know from the screenshot he posted for the Images Storage Groups. default is listed there. I’m guessing that’s what the logs are referring to.

                                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                      Daily Clean Installation Results:
                                      https://fogtesting.fogproject.us/
                                      FOG Reporting:
                                      https://fog-external-reporting-results.fogproject.us/

                                      E 1 Reply Last reply Apr 12, 2017, 12:32 AM Reply Quote 0
                                      • E
                                        EuroEnglish @george1421
                                        last edited by Apr 12, 2017, 12:07 AM

                                        @george1421

                                        Based on your thoughts about the Image replication log files I used Webmin to check the Images folder on each server. It seems that you might be correct, it shows that there is a small file size difference between the main server and the two storage servers. It also seems that rights didn’t transfer the same, both storage servers have different rights to the files.

                                        I marked each file manager with the server for reference:

                                        0_1491955601541_Fog File manager.jpg

                                        G 1 Reply Last reply Apr 12, 2017, 12:10 AM Reply Quote 0
                                        • G
                                          george1421 Moderator @EuroEnglish
                                          last edited by george1421 Apr 11, 2017, 6:12 PM Apr 12, 2017, 12:10 AM

                                          @EuroEnglish The other thing I see is file ownership is different on each system too. That’s not right.

                                          Fog will use the user account listed for each storage node to copy the files from the master node to the remote nodes. This is done over ftp from the master server in the storage group to all slave nodes in the same storage group. Systems defined in FOG but in a different storage group will be left out of the replication cycle.

                                          Did you seed the files on each storage node or did FOG do this?

                                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                          E 1 Reply Last reply Apr 12, 2017, 12:20 AM Reply Quote 1
                                          • 1
                                          • 2
                                          • 1 / 2
                                          1 / 2
                                          • First post
                                            20/31
                                            Last post

                                          212

                                          Online

                                          12.0k

                                          Users

                                          17.3k

                                          Topics

                                          155.2k

                                          Posts
                                          Copyright © 2012-2024 FOG Project