• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Deployment stuck at x percentage

    Scheduled Pinned Locked Moved Unsolved
    FOG Problems
    4
    24
    832
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • george1421G
      george1421 Moderator @sega
      last edited by

      @sega Ok the picture tells me more about where its hanging. When it hangs here it more indicative that you have a network infrastructure issue. At this point you are running under the FOS (linux) engine. You could always try to update the kernel to the latest 6.x version using the web ui fog configuration->kernel update. But my intuition is telling me its something outside of the fog ecosystem causing this outage.

      The transfer rate of 948MB/min is also not good. On a well behaved/designed 1GbE network you should see 5.7 to 6.5GB/min To translate that 948MB/min == 16.4MB/sec

      1GbE is 125MB/s
      100Mhz is 12.5MB/s

      With data compression 16.4MB/sec is pretty close to 12.5MB/s. So just using some wild guessing I would have to say you have somewhere between the FOG server and the target computer a 100Mb/s network link.

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

      S 1 Reply Last reply Reply Quote 0
      • S
        sega @george1421
        last edited by

        @george1421 On that place we have 10 pcs, but all are connected at a one gigabyte port. So when all are deplyoing at the same time, the rate is decreased. If just one is deploying the rate is at that 5.7 GB/min.
        But you saying that the problem isn’t something on the FOG more like outside, like the network connection?

        F george1421G 2 Replies Last reply Reply Quote 0
        • F
          fogcloud @sega
          last edited by

          @sega Have you used FOG in the past, and if so, did you encounter this problem back then? It would be helpful to know if this problem is something that has only occurred recently or if it has always been this was since using FOG. If something did change, we need to narrow down what it could be.

          When a computer gets “stuck,” does the data block percentage change at all? Have you tried leaving a computer to see if it finishes the imaging process even if it’s slow?

          Have you tried plugging in the computers to the same network switch that the FOG server is connected to? The would eliminate any other bottlenecks in network speed on your network.

          1 Reply Last reply Reply Quote 0
          • george1421G
            george1421 Moderator @sega
            last edited by

            So when all are deplyoing at the same time, the rate is decreased.

            Ah you are deploying more than one image in a push. Just be aware that you can saturate a single 1GbE link with between 2 and 3 simultaneous unicast image deployments. If you need to deploy 10 at a time them I would look at using a multicast image deployment. This will only send out 1 image with all 10 receiving that same image. Its much easier on the network.

            Also if you want to do this with unicast messages, upgrade your fog server network link to 10GbE. If that isn’t possible add 2-3 network ports connected in a network LAG (trunked) configuration. With multiple unicast deployments a LAG group will surely help.

            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

            S 1 Reply Last reply Reply Quote 0
            • S
              sega @george1421
              last edited by sega

              @george1421 The 10 pcs getting all the same image, but through the normal deployment. It would be still better to use multicast deploying in that case?

              Edit: I tried the multicast for one group now, they starting the Partclone part but don’t start the deplayoment. As I seen the FOG server and the client pcs has to be in the same VLAN for multicast? That’s not our case, they have different ones.

              george1421G 1 Reply Last reply Reply Quote 0
              • george1421G
                george1421 Moderator @sega
                last edited by

                @sega said in Deployment stuck at x percentage:

                As I seen the FOG server and the client pcs has to be in the same VLAN for multicast? That’s not our case, they have different ones.

                Generally the fog server and target computers should be on the same vlan for multicasting to work. It can work if you have a igmp-proxy server setup on your router between the vlans. Multicast traffic don’t normally pass a normal router.

                But now in your network you’ve introduced another choke point in your router must be able to handle the load of these 10 systems being imaged simultaneously. As I said you can flood a 1GbE link with between 2-3 simultaneous unicast images. Also on your fog server, is the image stored on what media ssd or hdd? If hdd, how many spindles are being used? That will have an impact on performance too if its hdd.

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                S 2 Replies Last reply Reply Quote 0
                • S
                  sega @george1421
                  last edited by

                  @george1421
                  We have the following structure:

                  • FOG Server on office VLAN
                  • 3 rooms with separate VLANs each (because they have different rights in the network, but all can access the FOG server)
                  • FOG server is running on the VM but on ssd, if I’m correct
                  • All are linked with aruba managed switches

                  Somewhere I read that it’s possible to have it with different VLANs if the udp stream is forwarded.
                  The weird thing is: we didn’t had the problems in the past… so yea something in the network could be the problem.

                  S 1 Reply Last reply Reply Quote 0
                  • S
                    sega @sega
                    last edited by sega

                    This post is deleted!
                    1 Reply Last reply Reply Quote 0
                    • S
                      sega @george1421
                      last edited by sega

                      @george1421

                      The current status: FOG and client pc in different VLANs. We have managed switches where the streams should be forwarded (we don’t managed them ourselves). We started a multicast session for 6 pcs. They are all going into the Partclone windows, but don’t start the deployment process.

                      In the active task tab I have once (under “Active Tasks”) 6 task (for each pc one) and under “Active Multicast Tasks” I have one for the whole group. State is in “In-Progess” and status is “0”.

                      edit: I somehow can’t post the logs because they are marked as spam

                      Tom ElliottT george1421G 3 Replies Last reply Reply Quote 0
                      • Tom ElliottT
                        Tom Elliott @sega
                        last edited by

                        @sega if you want more “speedy” unicast I would suggest setting your storage nodes queue limit to 2 or three.

                        I know this sounds low and all but it’d be better to have 10 systems imaged in 30 - 45 minutes than to have 10 systems imaged in 2-3 hours.

                        Unless you can get multicast to work, I think this is your best approach. Another method, though unrealistic, would be to setup multiple storage nodes in the same group (each with only 2 or 3 per vlan) and possible the location plugin to designate the vlan systems to their respective vlan ip groups.

                        Ultimately keeping things as they currently are is not going to magically work the next time you try again.

                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                        1 Reply Last reply Reply Quote 0
                        • Tom ElliottT
                          Tom Elliott @sega
                          last edited by

                          @sega if you can post logs to paste bin and send the link in the forums that may work as well.

                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                          S 1 Reply Last reply Reply Quote 0
                          • S
                            sega @Tom Elliott
                            last edited by

                            @Tom-Elliott

                            That’s a good idea to reduce the limit. But our priority will be to get multicast to work. I’m pretty sure somewhere in our network structure is the error why it’s not working yet.
                            Since we have to do that multiply times in a week, reducing the time would be great.

                            And here are the logs: https://pastebin.com/JQadkbZ0

                            1 Reply Last reply Reply Quote 0
                            • george1421G
                              george1421 Moderator @sega
                              last edited by

                              @sega Multicasting should work correctly if the fog server and target computers are on the same subnet. You said that may not be the case. For multicasting to work your router has to support forwarding multicast packets or your router or some device that has access to all of the vlans need to be running an igmp-proxy. This is akin to what a dhcp-helper/relay is to dhcp. You should see exactly what you are seeing if the network doesn’t support multicasting. The FOS Engine is started using unicast messaging, the partcone part is where the multicast stream really starts moving data.

                              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                              S 2 Replies Last reply Reply Quote 0
                              • S
                                sega @george1421
                                last edited by

                                @george1421

                                Yea, it’s not possible having the FOG and the target pcs on the same subnet. Our switches have the option active igmp. And I see a warning on the switches that they get a v3 query but the device is just configured for igmp v2. But that I have to look with our service provider.

                                1 Reply Last reply Reply Quote 0
                                • S
                                  sega @george1421
                                  last edited by

                                  @george1421 @Tom-Elliott

                                  We just saw our switches can’t process v3… Is it possible to change a setting that FOG uses the v2 of igmp?

                                  Tom ElliottT george1421G 2 Replies Last reply Reply Quote 0
                                  • Tom ElliottT
                                    Tom Elliott @sega
                                    last edited by

                                    @sega https://www.udpcast.linux.lu/hints.html

                                    I think this may tell you more?

                                    We don’t control what “version” your switches can/cannot allow?

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                    S 1 Reply Last reply Reply Quote 0
                                    • S
                                      sega @Tom Elliott
                                      last edited by

                                      @Tom-Elliott

                                      I mean is there a setting in FOG where I can change it or does FOG just uses the version that is available?

                                      Tom ElliottT 1 Reply Last reply Reply Quote 0
                                      • Tom ElliottT
                                        Tom Elliott @sega
                                        last edited by

                                        @sega I think that’s where I’m confused.

                                        ALl FOG does is start a software that sends packets. How those packets traverse your network is independent of that (from what I can tell).

                                        Layer 2/Layer 3.

                                        I think you’d want to look into something IGMP Snooping or something for V2 functionality?

                                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                        S 1 Reply Last reply Reply Quote 0
                                        • S
                                          sega @Tom Elliott
                                          last edited by

                                          @Tom-Elliott
                                          Alright, that was the point where I was confused too. Now I understood.

                                          1 Reply Last reply Reply Quote 0
                                          • george1421G
                                            george1421 Moderator @sega
                                            last edited by

                                            @sega I don’t know enough to answer your question intelligently.

                                            I can tell you that for multicast imaging FOG uses the linux commands udpsend and udpreceive.

                                            I don’t see any command line switches to pick which igmp mode is being used.
                                            https://users.soe.ucsc.edu/~kent/test/udpcast.html

                                            I can say its rare that there is ever a question on the revision of igmp being used.

                                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                            1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 1 / 2
                                            • First post
                                              Last post

                                            199

                                            Online

                                            12.0k

                                            Users

                                            17.3k

                                            Topics

                                            155.2k

                                            Posts
                                            Copyright © 2012-2024 FOG Project