Image Size of the client in Fog does not match the size of the Capture in Hard Drive

[rmishra1004]

OS: CentOS 7
Fog version : 1.5.8

Problem:

First Capture in Fog: Image size = 15.03 GiB

Second Capture in Fog: Image Size: 29.46 GiB

Third Capture in Fog: Image Size: 44.70 GiB

Host details after third capture: 16.4 GB

[george1421]

This is a bug that was discovered in 1.5.7, where each capture would take the previous value and add the new captured value to it, basically doubling or tripling the size depending on the number of image captures. That bug may have been reported too late to get into the 1.5.8 release.

At the end of the day, this value is only used for display and not for any functional use in FOG. It needs to be right, but is not a functional requirement.

@Developers we probably need to pin this to 1.5.9.

MOD Note: Moving to Bug reports.

[Sebastian Roth]

@rmishra1004 Thanks for reporting. We have another user but I wasn’t able to replicate the issue yet and didn’t get the information needed to find what is causing this. Cross linking the topics so we keep track of this: https://forums.fogproject.org/topic/14160/image-size-on-client-incorrect

Can you please post the contents of the text file d1.minimum.partitions here in the forums. You find that on your FOG server in /images/Win10LTS/ folder.

[dforce]

I have the same problem using version 1.5.7 and 1.5.8 (latest one).
My Linux partition of has the size of 500 Gb and over capturing and deploying the mage my Linux partition now has a size of 8.6 Gb.

Maybe this can be solved as quickly as possible ?

Thanks
Miguël

[Quazz]

@dforce I don’t think this is the same thing.

Sounds like you created a “resizable” image type, which will indeed shrink the main partition so that it can be deployed on a larger array of devices where it will then expand to full size (relative to disk size).

[dforce]

@Quazz
Problem is that the image/partition is not expanding to the full size after deploying. It worked in previous versions but from 1.5.7 until the latest version it just kept the schrinked/resized value.

This is just happening for 1 partition, all the others are expanding as they should do.

[Sebastian Roth]

@dforce Partition layouts are usually different and cause different behaviour. Can you please open a new topic and post your details. We need the contents of the text files d1.partitions, d1.minimum.partitions and d1.fixed_size_partitions!

[rmishra1004]

@Sebastian-Roth Please find the details below:

itadmin@fog120:~$ ll -lah /images/Win10LTS/
total 7.0G
drwxrwxrwx.  2 root       root  222 Feb 21 12:50 .
drwxrwxrwx. 21 fogproject root 4.0K Feb 21 12:50 ..
-rwxrwxrwx.  1 root       root    6 Feb 21 12:45 d1.fixed_size_partitions
-rwxrwxrwx.  1 root       root 1.0M Feb 21 12:45 d1.mbr
-rwxrwxrwx.  1 root       root  861 Feb 21 12:45 d1.minimum.partitions
-rwxrwxrwx.  1 root       root   15 Feb 21 12:45 d1.original.fstypes
-rwxrwxrwx.  1 root       root    0 Feb 21 12:45 d1.original.swapuuids
-rwxrwxrwx.  1 root       root 382M Feb 21 12:45 d1p1.img
-rwxrwxrwx.  1 root       root  11M Feb 21 12:45 d1p2.img
-rwxrwxrwx.  1 root       root 160K Feb 21 12:45 d1p3.img
-rwxrwxrwx.  1 root       root 6.6G Feb 21 12:50 d1p4.img
-rwxrwxrwx.  1 root       root  861 Feb 21 12:45 d1.partitions

itadmin@fog120:~$ cat /images/Win10LTS/d1.partitions
label: gpt
label-id: BCD2617A-B515-4B89-9A22-BC45CC4A85A9
device: /dev/sda
unit: sectors
first-lba: 34
last-lba: 498859998

/dev/sda1 : start=        2048, size=     1021952, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=2B82619D-55BC-4245-989F-5D9247BFA675, name="Basic data partition", attrs="RequiredPartition GUID:63"
/dev/sda2 : start=     1024000, size=      204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=30CAD430-414E-456F-9805-34FF535BB883, name="EFI system partition", attrs="GUID:63"
/dev/sda3 : start=     1228800, size=       32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=C2749A80-A11C-4BD0-BD6F-CF5433E61773, name="Microsoft reserved partition", attrs="GUID:63"
/dev/sda4 : start=     1261568, size=   497597952, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=B36C764C-00DC-40D9-A06A-ADA52A187F5A, name="Basic data partition"

itadmin@fog120:~$ cat /images/Win10LTS/d1.minimum.partitions
label: gpt
label-id: BCD2617A-B515-4B89-9A22-BC45CC4A85A9
device: /dev/sda
unit: sectors
first-lba: 34
last-lba: 498859998

/dev/sda1 : start=        2048, size=     1021952, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=2B82619D-55BC-4245-989F-5D9247BFA675, name="Basic data partition", attrs="RequiredPartition GUID:63"
/dev/sda2 : start=     1024000, size=      204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=30CAD430-414E-456F-9805-34FF535BB883, name="EFI system partition", attrs="GUID:63"
/dev/sda3 : start=     1228800, size=       32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=C2749A80-A11C-4BD0-BD6F-CF5433E61773, name="Microsoft reserved partition", attrs="GUID:63"
/dev/sda4 : start=     1261568, size=    32735296, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=B36C764C-00DC-40D9-A06A-ADA52A187F5A, name="Basic data partition"

itadmin@fog120:~$ cat /images/Win10LTS/d1.fixed_size_partitions
1:2:3

[Sebastian Roth]

@rmishra1004 Thanks for the detailed information. Finally was able to replicate the issue. Though it wasn’t really due to your specific layout it definitely helped to keep playing with the things and made me find it in the end.

@Tom-Elliott Turns out the commit on not resetting client size on check-in already is causing this. I haven’t been in the project long enough to actually know the client size is actually being set within the progress update (statusreporter). So it’s actually true, as soon as the actual data on the partitions changes this new number will be added eventually causing the numbers to build up.

While I would really like to change this and make the size on client to be reported only once at the end of uploading an image this would take a lot more work. So I’d vote for reverting the mentioned change to what it was before in FOG 1.5.x and live with the issue that client size will be reported as zero in case an upload doesn’t finish properly. That’s how it has been for ages and I don’t see how we could fix that without messing with things a lot more.

[Tom Elliott]

@Sebastian-Roth just thinking this through a bit, what about resetting and only the size during capture task. The route we took before would rewrite this data regardless of the task type. So a deploy could potentially reset the size to zero just as much as a capture.

So the reset, happens at capture task generation, not by the task checkin process.

And only update the size during a capture task as it’s moving along.

Maybe a better and less error prone method:

Read the images d1.minimum.partitions size (or d1.partitions if minimum doesn’t exist) and calculate the size from that? This method wouldn’t rely on progress updates and would, I think, be more accurate as well.

[Sebastian Roth]

@Tom-Elliott There are certainly various ways to do this. Though I would like to keep fixes for 1.5.x as simple as possible so we don’t break new things in that. We’ll surely rewrite this for 1.6.x I would say.

Let me know if you can think of a straight forward simple fix for 1.5.x.

[Sebastian Roth]

@dforce @rmishra1004 This is fixed in the latest dev-branch. Will be in the next release.

[Ben Acheff]

Sorry for my late reply, re: the other thread that I started on this. I don’t manage the hardware-side of the FOG server so i couldn’t get the file specified. I was also furloughed from work for some time, so I’ve missed the earlier updates.