Weird cloning issue with slow capture speed
-
So please bare with me as this is all new to me. I recently setup a Fog computer in my house so I can make images of all of computers in our house and so on. My setup is a followed as the fog server.
i5 650
4gb of Ram
1TB Image drive
Running the latest FOG
60 megabit internet connection
This is a older machine so it only has a 100mbps network card.The issue I’m having is that when I go to capture a image it starts out very fast around 6-7gb a minute and I notice the hard drive light is actually doing something on the client machine. Then after a minute or so the hard drive light goes completely out and then the speed of the capture goes as low as 500mb a minute and then once the Hard drive light indicates it doing something the speeds starts going up a lot. It goes on like this until it completes the capture.
This happens on all machines with a hard drive and SSD.
My settings of the image are as followed.
image url)I just saw this issue it seems related when I go to task Manager on the fog server the network is at 100% this is when the Hard Drive light goes off then when the network goes down to 0% or 50% the hard drive light comes back on. Can someone tell me whats going on is it because i’m on 100mbps network card.
-
Hi blong10,
I am in no way related to Fog project but see below my observation.
I currently testing Fog between two computers with 1Gbit network cards connected through 1Gbit switch.
Fog server captures Windows 10 Pro image, HDD has two partitions.
Fog identifies first partition (about 500MB) and make decision to not compress it, with second partition it compresses data (on client side if I understand it correctly) and sends over network. I observe sustained speed about 590MB/min and to capture Windows 10 Pro on 160 GB HDD takes about 51 minutes (CPU AMD Athlon X2 3800++ – low performance by modern standard).
When I deploy this image back to the Windows 10 machine it takes about 6 minutes 45 seconds.
My vision is that due required compression CPU of the host had to work hard and network speed at this time is no concern.
I would assume that in your case probably some data read into the buffer then they compressed (during this time you do not see network activity) and only then spilled into network wire.
Probably more knowledgeable people in this forum will give more detailed explanation of your observation.
Polar Bear
-
@blong10 said in Weird cloning issue with slow capture speed:
This is a older machine so it only has a 100mbps network card.
The issue I’m having is that when I go to capture a image it starts out very fast around 6-7gb a minute and I notice the hard drive light is actually doing something on the client machine.That 100Mb/s network card is going to kill your performance so to speak. With a 100Mb/s nic card you will convert that to MB/s or 12.5MB/s x 60seconds == 750MB/min. So in an ideal world you can only achieve 750MB/m, so while 500MB/m is a little on the slow side its not too far off the theoretical maximum of 100Mb/s.
One thing you have to remember is that the speed listed in partclone is a composite speed based on network throughput, decompression speed, and writing the image to disk. Having a slow CPU or slow network or slow hard disk drive on the target computer will knock this partclone speed down.
Almost all of the heavy lifting of image deployment is done by the target computer. The FOG server only serves the image from disk to the network adapter, and then manages the entire imaging process. If you wanted to you could run the FOG server on a rasperry pi. It doesn’t need a lot of resources, just a fast disk and a fast network adapter. A powerful CPU in the fog server is just a waste.
-
@Polar-Bear said in Weird cloning issue with slow capture speed:
I currently testing Fog between two computers with 1Gbit network cards connected through 1Gbit switch.
I’m just going to comment on a typical network. For image capture on a modern computer I would expect to see 2-3GB/m transfer rates for capture and then 6.0-6.5 deploy rates for a well run 1GbE network. The image capture and compression take the largest hit by CPU performance. Just for a point of reference on my network I can deploy a 25GB fat windows image in just a little over 4 minutes @ 6.2GB/m but this is to a modern target computer.
Fog identifies first partition (about 500MB) and make decision to not compress it, with second partition it compresses data (on client side if I understand it correctly) and sends over network. I observe sustained speed about 590MB/min and to capture Windows 10 Pro on 160 GB HDD takes about 51 minutes (CPU AMD Athlon X2 3800++ – low performance by modern standard).
The 500MB boot sector is so small that data compression won’t help much. The data partition will. Your capture rate is a bit low, but a lot is dependent on the target system’s CPU and drive speed. A slow HDD will kill your upload rate. Some of the older HDD only had a trasfer rate of 40-70MB/s.
-
And as long as I’m rambling on. As you see in the picture there are 2 viable image managers. One is the FOG standard of Partclone Gzip. The new image manager is Partclone zstd. I will tell you that zstd is slower on image capture than gzip, but it makes up for it in a smaller image size on the fog server as well as faster image deployment rates. With a slow network adapter as in the 100Mb/s I would change the capture mode over to Partclone zstd. This is because a smaller image size will take less time to transfer over a slow network connection.
As long as we are talking speed improvements. If you plan on deploying multiple unicast images at the same time, having a single spindle HDD in the FOG server will kill your deployment rates. In this case I would switch the FOG server hard drive over to an SSD because there are no moving parts in a SSD so head seek time is almost zero.
-
In my old computer I have 160GB HDD which should give read speed around 120MB/sec.
For a second we will not take into account CPU compressing data and we should come to a number of 7.2GB/minute.
Now let’s drop AMD Athlon X2 3800+ with performance index of 999 by Passmark and compression kills capture speed to a range of 580-650MB/min. (Note: I have chosen compression index of 8 instead default 6 for no obvious reason)
https://www.cpubenchmark.net/cpu.php?cpu=AMD+Athlon+64+X2+Dual+Core+3800%2B&id=77
This is my observation so far.
Polar Bear
-
@Polar-Bear said in Weird cloning issue with slow capture speed:
In my old computer I have 160GB HDD which should give read speed around 120MB/sec.
My intent is not do dispute your numbers, but based on personal experience I have not seen HDDs to often have 90MB/s transfer rates. If you have one that does clock in at 120MB/s (measured and not marketing wank) then hold on to it, because that is at the low end of SSD drives.
And in the FWIW bucket, the higher the compression index is the slower the image capture rate will be, because it take more CPU resources to compress the image tighter.
-
Well George, I do not use special software to measure performance of the HDD – for me numbers are coming from ‘hdparm -tT /dev/sd?’ tests on 775 socket.
Although I prefer WD drives in this case I got for good price Seagate 2TB SATA drive (Seagates was not so reliable in comparison with WD in my practical life).
Just for notice: I run Linux server with 4 WD Blue HDDs 2TB each in RAID 5 configuration and performance on read side is around 380MB/sec.
Polar Bear
-
@george1421
George,just for a test I decided to capture image with compression index 6 (default) and average speed was 1GB/minute (sometimes hit 1.2GB/min).
If before to capture image with compression index 8 it was taking 51 minutes, now with compression index 6 it takes close to 28 minutes.
Polar Bear
-
@Polar-Bear said in Weird cloning issue with slow capture speed:
1GB/minute (sometimes hit 1.2GB/min)
For index of 6 I would like to say that’s on the low end of normal for about a 3 year old hardware (ssd but not nvme drives). Again its very subjective. The sweet spot for zstd I think is compression 11. But I would not be too concerned about the compression side because that happens only once. Its the deployment side where you really need the speed. You will get the fastest deployment speed from an uncompressed image, but an uncompressed image will take the most space on your fog server and consume the most bandwidth on your network. So you need to decide on the trade offs and pick the best compression factor for your organization.
-
@george1421
George,this test computer is more than 10 years old when Windows XP still was a major Microsoft OS
https://support.hp.com/ca-en/document/c01680017
an article for a date reference
https://www.pcworld.com/article/132004/article.html
I found this computer disposed and decided to give it a second life, yesterday it misbehaved - I looked at mainboard and found two bulged capacitors – nothing what solder iron can not fix, few minutes and little termo-paste and computer runs again.
Half an hour to capture image is not too bad…
Polar Bear
-
So just a quick update. I found a 1gagibit adapter for my desktop and it is giving me 1gbps in network management tab. The clone seems much better now when capturing my network usage is no longer 100%. HOWEVER I’m having the same issue where the Hard drive light goes on and off constantly. it goes faster when it kicks in.Right know I’m capturing a 250SSD and it’s going to take around 1hr at 2.76GB/min. Is this normal. And can someone explain the Hard drive going on and off is there a way to make it stay consistently on becuase it seems like that woud speed it up a lot Thanks guys!!
-
@blong10 I may have not said this in this thread, but imaging is like a 3 legged stool. You have the CPU+memory, network, disk adapters. They are all part of imaging. They all have an interconnecting relationship to imaging speed. Since your disk light is going on and off and is not a constant stream, i would say that your bottleneck is either with network or more likely your CPU+memory. When FOG images (upload in this case) it read a block from your disk subsystem into memory, compresses it and then sends it on to the network and FOG server. Since your disk light is going on and off at a slow pattern we might guess that your disk subsystem is waiting for the other 2 legs to do their parts. If I had to guess on a 10 year old system it would be CPU+memory where your bottleneck is in your system.
Again I wouldn’t be so concerned about upload speeds because that is typically a one time event. The deployment is where you need/want the speed because that is the many times event. I’ll repeat this, the target computer has the most impact on imaging speeds because it does all of the work in the imaging process.
-
@george1421 Ok thnak you for this. Just to clarify the hard drive light that goes on and off is not on the client machines that i capture the image from. I’ll try to do a deployment tomorrow of the image and see how it goes. I also forgot to mention I have 4gb of DDR3 running on the Fog Computer. And I’m running VM ware in the background for fog. Right now my memory usage is around 78-80% and My Fog VM was 2gb that I gave it . Do you think would play a role in it not having enough RAM.
-
@blong10 The fog server really doesn’t do a lot in the imaging process. I would give it at least 4GB of ram to release some of the memory constraints though. The only thing the fog server does is move the image from the network to the disk. It doesn’t process the image at all. Depending on the number of FOG Clients you have a 2 vCPU VM with 4GB of RAM is sufficient.
