• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Another slow Deployment qustion

    Scheduled Pinned Locked Moved
    FOG Problems
    6
    41
    12.9k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Wayne WorkmanW
      Wayne Workman @george1421
      last edited by Wayne Workman

      @george1421 I didn’t suggest one at a time, I suggested two at a time.

      I can test this in the lab and give hard numbers, I know this because I’ve had max clients set to 10 before and it was terrible, and changed it to 2 and now it’s performing great.

      Also, pretty sure the figures Partclone displays is read/write speed, not network transfer speed.

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
      Daily Clean Installation Results:
      https://fogtesting.fogproject.us/
      FOG Reporting:
      https://fog-external-reporting-results.fogproject.us/

      1 Reply Last reply Reply Quote 0
      • Wayne WorkmanW
        Wayne Workman @george1421
        last edited by Wayne Workman

        @george1421 Take this video for example, look at the video @ 5:34

        We see the space used on the image is 31.7, write speed is 2.29GB/min.

        We see elapsed time is 13 minutes and 31 seconds.

        31.7 divided by 2.29 = 13.84, or 13 minutes and 50 seconds.

        Therefore the rate that Partclone displays is write speed (or read speed), not network transfer rate. The image is sent across the network in compressed form, so there is much more room available on the network. Once network saturation is reached, it’s reached. However, if you continue to pile on imaging tasks, eventually the HDD seek time is maxed, and you actually start loosing network saturation. This is my reasoning for setting Max Clients to 2 or 3, not 10.

        The idea is to saturate the available bandwidth for imaging tasks, but not exceed the HDD’s seek ability to keep up. If you exceed the HDD’s ability to keep up due to seek times, then you’re going backwards.

        The optimal setting for max clients should saturate the network fully, but not exceed the HDD’s ability to seek and keep up with the full network speeds.

        https://www.youtube.com/watch?v=OgKwhuNB1cs

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
        Daily Clean Installation Results:
        https://fogtesting.fogproject.us/
        FOG Reporting:
        https://fog-external-reporting-results.fogproject.us/

        george1421G 1 Reply Last reply Reply Quote 0
        • A
          Arsenal101
          last edited by

          Great replies guy! I thought that the rate was always the network transfer speed and not the write speed…

          @Wayne-Workman I am in now way discrediting your method. I just don’t think its the best set up for the needs that we have right now. I want to stay away from multicast for now as we dont have out layer 3 switch/router configured to pass broadcast packets as we have had several issues with staff Looping the network and bringing down everything instead of just one building (this was also before we turned spanning tree on), and I know my boss wont be a fan of increasing the broadcast domain even if was just for the summer.

          on top of that we don’t have the time to build more storage nodes for fog so it’s kind of a trade off yes they image slower but at the same time the you’re doing 10 at a time per storage node (20 at a time in my case 1 master + 1 storage) and 800MB-PM is plenty acceptable for what we need it for.

          @george1421 Maybe at some point I could convince my boss to throw a 10gb NIC and some decent hardware for our fog server in the budget. I could try some of the stuff you are suggesting. and that would really boost the speed right up.

          If we could ever get it 10gb to the end PC the only thing slowing the process down would be the human factor!

          Wayne WorkmanW george1421G 2 Replies Last reply Reply Quote 0
          • Wayne WorkmanW
            Wayne Workman @Arsenal101
            last edited by

            @Arsenal101 I was not suggesting multicast? But ok.

            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
            Daily Clean Installation Results:
            https://fogtesting.fogproject.us/
            FOG Reporting:
            https://fog-external-reporting-results.fogproject.us/

            1 Reply Last reply Reply Quote 0
            • A
              Arsenal101
              last edited by

              @Wayne-Workman Sorry I concatenated @Sebastian-Roth’s and your reply in my head.

              1 Reply Last reply Reply Quote 0
              • george1421G
                george1421 Moderator @Arsenal101
                last edited by

                @Arsenal101 said in Another slow Deployment qustion:

                If we could ever get it 10gb to the end PC the only thing slowing the process down would be the human factor!

                While 10G to the desktop would be really nice, its not necessary and a bit of a waste because I would suspect on the target end, the disk or CPU is your limiting factor and not the network. For the server managing multiple data streams I can see the network and then the disk subsystem being the bottleneck.

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                A 1 Reply Last reply Reply Quote 0
                • A
                  Arsenal101 @george1421
                  last edited by

                  @george1421 Pretty much, It would just be cool to say we have 10gb to the desktop… A SSD drive would be sweet though!

                  1 Reply Last reply Reply Quote 0
                  • george1421G
                    george1421 Moderator @Wayne Workman
                    last edited by

                    @Wayne-Workman said in Another slow Deployment qustion:

                    @george1421 Take this video for example, look at the video @ 5:34

                    We see the space used on the image is 31.7, write speed is 2.29GB/min.

                    We see elapsed time is 13 minutes and 31 seconds.

                    31.7 divided by 2.29 = 13.84, or 13 minutes and 50 seconds.

                    Therefore the rate that Partclone displays is write speed (or read speed), not network transfer rate.

                    Wayne I fully agree with you and understand that the part clone display is not actual network usage, but its the best metric we have without getting into to much tech. So the point it it’s not accurate but its the best we have (like the Windows Performance Index, at least its some metric that we can use as a baseline).

                    My testing with the intel nuc as the FOG server and deploying to a e6400 with a HHD and SSD. You can see the speed difference in just switching the target from a HDD to SSD with everything else being the same.

                    https://forums.fogproject.org/topic/6373/fog-mobile-deployment-server-for-200-usd-finished/3

                    “I replace the seagate rotating hard drive in the e6400 with a Crucial MX100 256GB SSD I had laying around. I again redeployed the same image as in the previous tests, this time the transfer rate was 7.8GB/m (130MB/s {!!faster than wire speed!!} ) according to partclone. As compared to 5.1GB/m with a rotating disk in the target. I booted the e6400 back into debug mode and ran hdparm -Tt /dev/sda hdparm reported 242MB/s for buffered disk reads as compared to 80MB/s with rotating media.”

                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                    Wayne WorkmanW 1 Reply Last reply Reply Quote 0
                    • Wayne WorkmanW
                      Wayne Workman @george1421
                      last edited by

                      @george1421 We can get a exact metric if we turn on FTP_Image_Size on the server, it’ll display the image size. Then we can use total time elapsed during imaging to calculate throughput.

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                      Daily Clean Installation Results:
                      https://fogtesting.fogproject.us/
                      FOG Reporting:
                      https://fog-external-reporting-results.fogproject.us/

                      Tom ElliottT 1 Reply Last reply Reply Quote 0
                      • Tom ElliottT
                        Tom Elliott @Wayne Workman
                        last edited by

                        @Wayne-Workman Throughput is already displayed on the dashboard (granted not per host), but if the network is NOT saturated as this is trying to lead to, you should see “plenty” of available bandwidth of the network there.

                        The part that bothers me is this still feels more like a networking issue than a seek/io issue. While I do totally understand IO as being a part of this, your network is most likely the first culprit. Primarily considering the mount point is used across the network to begin with.

                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                        1 Reply Last reply Reply Quote 1
                        • S
                          Sebastian Roth Moderator
                          last edited by

                          @Arsenal101 said:

                          Maybe at some point I could convince my boss to throw a 10gb NIC and some decent hardware for our fog server in the budget.

                          As George already said this is not very wise. I might add that multicast would solve all that I am pretty sure. I don’t understand why everyone is so afraid to get multicast running?? What’s that drawback that I don’t seem to see? Please tell me…

                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                          Q 1 Reply Last reply Reply Quote 0
                          • Q
                            Quazz Moderator @Sebastian Roth
                            last edited by

                            @Sebastian-Roth Only downside is reduced flexibility, that is to say if you have 20 computers and they need 7 different images, multicast won’t really be that useful.

                            Other than that, multicast is pretty great. Is there any word on the state of the torrent mechanic or is that on hold/abandoned? I personally think multicast would be better than the torrenting from a network saturation/resources perspective, but maybe I’m wrong?

                            1 Reply Last reply Reply Quote 0
                            • S
                              Sebastian Roth Moderator
                              last edited by

                              @Quazz said:

                              Only downside is reduced flexibility, that is to say if you have 20 computers and they need 7 different images, multicast won’t really be that useful.

                              While you are right about the impossibility to deploy different images via multicast I don’t see this as a fair argument. A network/switch can handle unicast and multicast at the same time so it’s not an either or thing. Use whichever is appropriate for the task you want to run. Multicasting when you actually want different images is stupid and unicasting when you want the same image is not very wise either.

                              Other than that, multicast is pretty great. Is there any word on the state of the torrent mechanic or is that on hold/abandoned? I personally think multicast would be better than the torrenting from a network saturation/resources perspective, but maybe I’m wrong?

                              There has been a discussion on and off on the forum. Just search for ‘torrent’ and I am sure you’ll find it.

                              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                              1 Reply Last reply Reply Quote 1
                              • Wayne WorkmanW
                                Wayne Workman
                                last edited by

                                While I still stand by what I’ve already said - just try multicast and see if it works - for same images at once. Stick computers into a group, and initiate multicast from the group.

                                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                Daily Clean Installation Results:
                                https://fogtesting.fogproject.us/
                                FOG Reporting:
                                https://fog-external-reporting-results.fogproject.us/

                                1 Reply Last reply Reply Quote 0
                                • 1
                                • 2
                                • 3
                                • 2 / 3
                                • First post
                                  Last post

                                212

                                Online

                                12.0k

                                Users

                                17.3k

                                Topics

                                155.2k

                                Posts
                                Copyright © 2012-2024 FOG Project