Image upload & deploy taking a long time

[george1421]

@brad-schumann Ok that speed isn’t bad, you are running a slightly higher compression than default so that will slow down throughput a little.

I guess we need to get the @developers comments on why would imaging switch from single disk resizable to raw. I know there are reasons that FOG will do this automatically, I just don’t know off the top of my head.

[edit] since this is a 500GB drive, can we assume its a SATA traditional hard drive?

[Sebastian Roth]

Either image type has been set to raw (by hand?) in the web UI. Please check the image settings there. If not then I think the upload scripts fail to recognize the filesystem on disk. Can you please schedule a debug (like scheduling a normal job but just before clicking the button there is a checkbox for debug) upload job next and when you get to the shell run the following command, take a picture and post here: blkid

[Brad Schumann]

@sebastian-roth @developers here are 2 screenshots.

[jhalbert]

From my use of FOG for the short about of time. Few key important parts for making capturing image go faster and deploying. I know when I skip these steps, FOG will capture much more. Which takes longer.

Step one - Turn off hibernate mode and delete the file.
Step two - turn off swap file and delete the file
Step three - Delete all cache files
Step four - defrag the drive
Step five - turn on hibernate and swap
Step six - capture the image. I am running raid 10 with 10 gig network. Windows 10 fully updated and ready to work takes about 45 minutes. To deploy takes about 5 minutes. My image size is about 50 gb.

[x23piracy]

Hi,

i would suggest to test both over netio: https://web.ars.de/netio/
It’s easy, one binary, can act as server and client, look at the syntax:

Speed issue can have several causes, so knowing the native speed would be a good information in first instance.
Even strong compression/decompression actions can hit the brake.

Please post your results

Regards X23

[Sebastian Roth]

@Brad-Schumann Please boot up the client into another debug task and run the following commands:

blkid -po udev /dev/sda1 | awk -F= /FS_TYPE=/'{print $2}'
blkid -po udev /dev/sda2 | awk -F= /FS_TYPE=/'{print $2}'
blkid -po udev /dev/sda3 | awk -F= /FS_TYPE=/'{print $2}'
blkid -po udev /dev/sda4 | awk -F= /FS_TYPE=/'{print $2}'

Again take a picture of the output and post here. As well could you please take a picture of the full screen when partclone (blue text window) is doing the work. I need to know which partition it is working on. To me it kind of looks like it’s doing a raw capture of sda (the whole 500GB disk) which should not happen when it’s set to do resizable.

[Brad Schumann]

@sebastian-roth 

[Sebastian Roth]

@Brad-Schumann Well that’s interesting. The issue is with sda3 (and maybe sda2 as well) but not the whole disk. But it’s kind of surprising that it’s doing roughly 500GB on sda3. Definitely something wrong with the partitioning I’d suspect.

it used to take 10-15 min to image when I was on fog ver. 1.2 I upgraded to 1.4.4 and now it was taking roughly 4 hours to image.

Yeah, if I remember correctly filesystem discovery was a bit different in 1.2.0 and possibly you were just lucky. But seems like the new FOG version doesn’t like it.

Please post the contents of the files /images/TestLat5500/d1.partitions, /images/TestLat5500/d1.minimum.partitions and /images/TestLat5500/d1.fixed_size_partitions

[Brad Schumann]

@sebastian-roth this is happening on all our new (straight from box) Dell Latitude 5580’s

[Sebastian Roth]

@brad-schumann said in Image upload & deploy taking a long time:

this is happening on all our new (straight from box) Dell Latitude 5580’s

Sure because something in the image meta data is not working out. So deploying the same image will always result in the same issue on the same kind of hardware. What I mean is that the image (probably being captured back on the FOG 1.2.0 server, right?) is not playing nicely when trying to deploy from a 1.4.4 server. So we need to figure out what’s wrong with it. Please post the information as requested and I will look into it.

[Brad Schumann]

@sebastian-roth Right I am tracking… here at the files:

[Sebastian Roth]

@Brad-Schumann Ok sorry, I think I looked at it the wrong way for a while. Had to turn my head crossways to get a clear view.

So it looks as if d1p2.img (sda2) and d1p3.img (sda3) of that image on the server are in RAW mode! Not sure why that is but surely you will always see it deploy as RAW if the image is like that. So I need to ask you when did you last upload/capture that image from your master client? Was this back in the FOG 1.2.0 times or more recently? Could you please schedule a debug upload task on your master machine that you want to capture from and run the following commands and post a picture of the output here:

blkid -po udev /dev/sda2
blkid -po udev /dev/sda3

Possibly FOG 1.2.0 just assumed NTFS filesystem when capturing windows systems and therefore didn’t have a problem. But as I said FOG 1.4.4 tries to figure out which filesystem you have on the partitions and uses the appropriate type then. In your case it’s unable to figure that it’s NTFS and uses RAW instead.

[Brad Schumann]

@sebastian-roth this originally create on the newer 1.4.4.

[Sebastian Roth]

@Brad-Schumann Sorry to ask again but this is from the source master system, right?

And this is Windows 10/7/Vista?

[Brad Schumann]

@sebastian-roth No worries, this is from the the source 5580 that we created (at least that is what my PC team tells me )
… and it is Win 10 64bit. I can re-create the image with a fresh version of windows on the master again and re-upload it to my Fog server, a

[Sebastian Roth]

@Brad-Schumann No, don’t recreate it yet. Possibly we’ll figure it out. Seems like we need to dig into the udev stuff to see why it doesn’t recognize the filesystem at all. Please post a picture of the output when running this command: udevadm info --query=all --name=sda3

[Brad Schumann]

@sebastian-roth

[george1421]

@brad-schumann I would like to give you a hint and a request as long as you have the FOS engine running there.

First the hint.
At the FOS command prompt (as in the picture) do the following

1. Key in ip addr show and note the IP address of the FOS target computer.
2. Give root a temporary password by keying in passwd then give it a simple password like hello This password will be lost when FOS reboots, so don’t worry.
3. Now from a windows computer connect to the FOS engine at the address gleaned at step 1 using putty (3rd party windows ssh application)
4. Login with root and the password you defined in step 2.

You can use putty to copy and paste stuff into the FOS engine from windows. It makes it easier to key stuff in and post to the forums because you can cut the text directly off the putty window and paste here.

Now for my request (if Sebastian already asked for this you can ignore). Key in the following at the command prompt

1. fdisk /dev/sda
2. At the fdisk prompt key in p
3. Copy the contents and paste it here.
4. At the fdisk prompt key in q and you will be back at the FOS command prompt.

What I’m interested in is the Id value in the 6th column.

[Brad Schumann]

@george1421 I didn’t see a 6th column below (assuming you meant row) is what I got.

Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 8897879E-4ECF-42A1-A6DC-F682422F50B6

Device                Start            End         Sectors      Size   Type
/dev/sda1              2048        1026047         1024000      500M   EFI System
/dev/sda2           1026048        1288191          262144      128M   Microsoft reserved
/dev/sda3           1288192      975800319       974512128    464.7G   Microsoft basic data
/dev/sda4         975800320      976637447          837128    408.8M   Windows recovery environment

[george1421]

@brad-schumann just for clarity this is what I expected to see.

Disk /dev/sda: 999.7 GB, 999653638144 bytes, 1952448512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0004a2f3

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     4098047     2048000   83  Linux
/dev/sda2         4098048  1952448511   974175232   8e  Linux LVM

It just may be the difference in the versions of fdisk program. The Id column tells what the partition ID is marked as. It was not a wasted effort since we have been able to collect more info on your exact hardware.

I am bit intrigued that the Type in your screen shot says “Microsoft basic disk”