Climbing Imaging Speed
-
Just wondering whether anyone has climbing imaging speed when doing images?
When I start an image it will begin at around 5GB/min but the longer it goes the faster it gets, up to around 10-15GB/min. I don’t exactly think it’s peaking as I had two running two images side by side, one towards the end doing 15GB/min and the other doing 5-6GB/min which has it over 20GB/min and once the first one stopped it’s not as if the other one instantly jumped up, just a steady climb.
I seem to recall this happening with my other FOG set up in the past, not exactly an issue as it does the job I was just more curious as to why it does this and whether everyone experiences this or just myself?
-
Your situation is rather unique. We typically see a peak transfer at the beginning of the imaging and then it settles down to a lower transfer rates as the buffers fill. If errors are detected in the transfer rate then rate limited starts to kick in and slow down the data stream even more.
I can’t think of a reason why it would start out slow and then raise during the transfer.
The thing to remember here that transfer rate is actually not network bandwidth but the decompression rate of the image at the client. On a 1 GbE network I would expect to see about 6.2GB / minute sustained transfer rates. A 10G network will of course transfer faster. The speed number provided by part clone is a composite number of how fast the image can be transferred from the FOG server over NFS, how fast it is received by the target computer, how fast the target computer can decompress the image, and finally now fast the target computer can write the decompressed image to disk. I might expect a modern quad core computer writing to ssd or nvme drive to get 10-14GB/min rates.
Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
-
It’s possible the network admin has implemented some sort of traffic shaping on your network that is causing this behavior.
-
@george1421 said in Climbing Imaging Speed:
The thing to remember here that transfer rate is actually not network bandwidth but the decompression rate of the image at the client.
It’s average write speed to disk.
Maybe we should just patch part-clone it so it shows actual write speed to disk & network transfer rate and also CPU utilization. -
@wayne-workman said in Climbing Imaging Speed:
It’s possible the network admin has implemented some sort of traffic shaping on your network that is causing this behavior.
I don’t think that’s the case as I’m able to move a file from the FP server down to my computer at 60-110MB/s so I don’t think there’s any shaping of that kind on the network.
It’s not really an issue though, I was just more curious as to why it does it as it does and whether it was a common thing or not which it doesn’t appear to be.
-
@dylan123 said in Climbing Imaging Speed:
I don’t think that’s the case as I’m able to move a file from the FP server down to my computer at 60-110MB/s so I don’t think there’s any shaping of that kind on the network.
Traffic shaping can be implemented on protocol/port basis. So transferring files through let’s say SMB might work at full speed but NFS could be slow.
Then as well there is one question not being asked as far as I can see. Is this imaging multicast or unicast and how many clients at any one time?
-
@sebastian-roth said in Climbing Imaging Speed:
@dylan123 said in Climbing Imaging Speed:
I don’t think that’s the case as I’m able to move a file from the FP server down to my computer at 60-110MB/s so I don’t think there’s any shaping of that kind on the network.
Traffic shaping can be implemented on protocol/port basis. So transferring files through let’s say SMB might work at full speed but NFS could be slow.
Then as well there is one question not being asked as far as I can see. Is this imaging multicast or unicast and how many clients at any one time?
Fair call, I’d like to think there’s no shaping though due to the fact it’s only myself and my manager in the IT team and we haven’t really had the need to shape traffic. I suppose it could be been set prior to us but I think it’s unlikely.
I believe I’m doing a unicast image (since I haven’t used the multicast image tab), typically it’s just the one client however I kicked off two not that far apart just to see if the speed would drop back on the first but that continued to rise whilst the first one started at about 5/6, but that’s more a slow climbing to begin with so I couldn’t see if both were capable of hitting 15+ together or if/when they’d peak.
-
@dylan123 I think this is very hard to debug not sitting right in front of the machines myself. You need to know that everything is this chain could be the bottle neck. So the best you can do is cross test things to hopefully nail it down. That means testing at least on three different clients in the exact same scenario (same network switch, no other things running over the network or on the FOG server at the same time). See if they all behave the same. If so, then you need to look into network or FOG server to find the issue. If they all go different speeds, then note down the numbers (start, end, average speed), redo the test again and see what you get. Possibly you can nail down one host that is different than the other two. Do a badblocks and memtest on that machine. As well see if that one has slightly different hardware.
