Image upload & deploy taking a long time

[george1421]

@brad-schumann said in Image upload & deploy taking a long time:

When I am uploading or deploying an image it takes 4-5 hours.

Imaging only use to take 10-15 min.

The confusion for me is here. Does it take 4-5 hours to image or 10-15 minutes. I can say for my reference image that is about 25GB in size it take about 4 minutes to push it to the target hard drive. If you based your transfer rate on the Partclone speed that is about 6GB/min.

[Brad Schumann]

@george1421 In Image Management it shows 465.30 under “Image size: on Client”.

[Sebastian Roth]

Well then the client’s disk is very full…

[Brad Schumann]

@george1421 it used to take 10-15 min to image when I was on fog ver. 1.2 I upgraded to 1.4.4 and now it was taking roughly 4 hours to image.

[george1421]

@brad-schumann This is only in regards to the speed issue: Can you tell us what the target computer you are using (mfg and model) as well as when you deploy the image what transfer rate is Partclone indicating? Is it the same as in your OP 5580 dell?

[Brad Schumann]

@sebastian-roth full of what? we only are using 56GB of the 500GB available on the hard drive. but Fog seems like it is coping the whole HD and what used to seem it would only image what was being used on the HD, or am I mis-understanding…

[george1421]

@brad-schumann said in Image upload & deploy taking a long time:

@george1421 In Image Management it shows 465.30 under “Image size: on Client”.

It almost sounds like fog is copying the file as RAW for some reason. Can you confirm from inside windows that the disk contains only a little percentage of the entire size?

[george1421]

@sebastian-roth Could we tell from the image meta data in the /images/<image_name> directory if fog was using raw or single disk non-resizable to capture the image?

[Brad Schumann]

@george1421 yes it is a Dell latitude 5580 laptop

[Brad Schumann]

@george1421 on the laptop being imaged currently the “File System” says raw. I have attached a pic.

[george1421]

@brad-schumann I think we found our speed issue. What I’m also interested in is just off the screen to the right. What is partclone saying for transfer rate?

[Brad Schumann]

@george1421 transfer speed is avg. 4GB/min, sorry there was a size limit on the jpeg upload.

[george1421]

@brad-schumann Ok that speed isn’t bad, you are running a slightly higher compression than default so that will slow down throughput a little.

I guess we need to get the @developers comments on why would imaging switch from single disk resizable to raw. I know there are reasons that FOG will do this automatically, I just don’t know off the top of my head.

[edit] since this is a 500GB drive, can we assume its a SATA traditional hard drive?

[Sebastian Roth]

Either image type has been set to raw (by hand?) in the web UI. Please check the image settings there. If not then I think the upload scripts fail to recognize the filesystem on disk. Can you please schedule a debug (like scheduling a normal job but just before clicking the button there is a checkbox for debug) upload job next and when you get to the shell run the following command, take a picture and post here: blkid

[Brad Schumann]

@sebastian-roth @developers here are 2 screenshots.

[jhalbert]

From my use of FOG for the short about of time. Few key important parts for making capturing image go faster and deploying. I know when I skip these steps, FOG will capture much more. Which takes longer.

Step one - Turn off hibernate mode and delete the file.
Step two - turn off swap file and delete the file
Step three - Delete all cache files
Step four - defrag the drive
Step five - turn on hibernate and swap
Step six - capture the image. I am running raid 10 with 10 gig network. Windows 10 fully updated and ready to work takes about 45 minutes. To deploy takes about 5 minutes. My image size is about 50 gb.

[x23piracy]

Hi,

i would suggest to test both over netio: https://web.ars.de/netio/
It’s easy, one binary, can act as server and client, look at the syntax:

Speed issue can have several causes, so knowing the native speed would be a good information in first instance.
Even strong compression/decompression actions can hit the brake.

Please post your results

Regards X23

[Sebastian Roth]

@Brad-Schumann Please boot up the client into another debug task and run the following commands:

blkid -po udev /dev/sda1 | awk -F= /FS_TYPE=/'{print $2}'
blkid -po udev /dev/sda2 | awk -F= /FS_TYPE=/'{print $2}'
blkid -po udev /dev/sda3 | awk -F= /FS_TYPE=/'{print $2}'
blkid -po udev /dev/sda4 | awk -F= /FS_TYPE=/'{print $2}'

Again take a picture of the output and post here. As well could you please take a picture of the full screen when partclone (blue text window) is doing the work. I need to know which partition it is working on. To me it kind of looks like it’s doing a raw capture of sda (the whole 500GB disk) which should not happen when it’s set to do resizable.

[Brad Schumann]

@sebastian-roth 

[Sebastian Roth]

@Brad-Schumann Well that’s interesting. The issue is with sda3 (and maybe sda2 as well) but not the whole disk. But it’s kind of surprising that it’s doing roughly 500GB on sda3. Definitely something wrong with the partitioning I’d suspect.

it used to take 10-15 min to image when I was on fog ver. 1.2 I upgraded to 1.4.4 and now it was taking roughly 4 hours to image.

Yeah, if I remember correctly filesystem discovery was a bit different in 1.2.0 and possibly you were just lucky. But seems like the new FOG version doesn’t like it.

Please post the contents of the files /images/TestLat5500/d1.partitions, /images/TestLat5500/d1.minimum.partitions and /images/TestLat5500/d1.fixed_size_partitions