Error trying to restore GPT partition deploying an Image to Smaller Disk

[scosta]

Hello guys, can you give some insight about this issue I’m having please?

We’re upgrading our park to Windows 10, so we prepared an Image using a Notebook with a disk of 250Gb. The image has 2 data partitions (one with 60Gb for the SO, and another for data with the rest of the disk).

The capture process was successful. The deploying in other hardwares with bigger disks went well too. The issue is when we try to deploy to a hardware with smaller disk, especificaly one with a 120GB SSD. We’re getting this error:


I’ll try to provide as much information as possible firsthand. Anything else you guys need, or tests you want me to run, just ask! Thank you very much for your time!

Scenario: Fog version 1.5.5
Details about image captured:

• Windows 10
• GPT,
• Two Partitions with effective data: 1) S.O., must always have 60GB, and its using about 30GB of effective data; and 2) Data, should occupy the rest of disk free space, this one is using about 6GB of effective data.
• Hardware where image was captured: Notebook with hard drive of 250GB
• Image Name: W10_SEMSIS_MULTI_1.01
• Image Path: /images/W10_SEMSIS_MULTI_1.01
• Image Type: Single Disk Resizable
• Partition: Everything
• Compression: 22
• Image manager: partclone gzip

Contents of /images/W10_SEMSIS_MULTI_1.01

ls -al /images/W10_SEMSIS_MULTI_1.01/
total 23558836
drwxrwxrwx  2 root root        4096 May  2 17:12 .
drwxrwxrwx 47 fog  root        4096 Apr 30 14:33 ..
-rwxr-xr-x  1 root root           3 May  2 14:56 d1.fixed_size_partitions
-rwxrwxrwx  1 root root     1048576 Apr 30 14:18 d1.mbr
-rwxrwxrwx  1 root root         738 May  2 14:51 d1.minimum.partitions
-rwxrwxrwx  1 root root          45 Apr 30 14:18 d1.original.fstypes
-rwxrwxrwx  1 root root           0 Apr 30 14:18 d1.original.swapuuids
-rwxrwxrwx  1 root root         311 Apr 30 14:18 d1.original.uuids
-rwxrwxrwx  1 root root      557796 Apr 30 14:18 d1p1.img
-rwxrwxrwx  1 root root    13404976 Apr 30 14:18 d1p2.img
-rwxrwxrwx  1 root root 19562007460 Apr 30 14:31 d1p3.img
-rwxrwxrwx  1 root root  4547186410 Apr 30 14:33 d1p4.img
-rwxrwxrwx  1 root root         738 May  2 14:49 d1.partitions

Content of d1.fixed_size_partitions

cat d1.fixed_size_partitions
:2

Content of d1.minimum.partitions

cat d1.minimum.partitions
label: gpt
label-id: CBDC4619-6786-11E9-97EA-F82201FA303B
device: /dev/sda
unit: sectors
first-lba: 34
last-lba: 500118158

/dev/sda1 : start=        2048, size=      751104, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=CBDC4615-6786-11E9-97EA-F82201FA303B, name="attrs=\x22RequiredPartition GUID:63"
/dev/sda2 : start=      753152, size=      202752, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=CBDC4616-6786-11E9-97EA-F82201FA303B, name="attrs=\x22GUID:63"
/dev/sda3 : start=      955904, size=   125829120, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=CBDC4617-6786-11E9-97EA-F82201FA303B
/dev/sda4 : start=   126785024, size=    13564664, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=CBDC4618-6786-11E9-97EA-F82201FA303B

content of d1.partitions

cat d1.partitions
label: gpt
label-id: CBDC4619-6786-11E9-97EA-F82201FA303B
device: /dev/sda
unit: sectors
first-lba: 34
last-lba: 500118158

/dev/sda1 : start=        2048, size=      751104, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=CBDC4615-6786-11E9-97EA-F82201FA303B, name="attrs=\x22RequiredPartition GUID:63"
/dev/sda2 : start=      753152, size=      202752, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=CBDC4616-6786-11E9-97EA-F82201FA303B, name="attrs=\x22GUID:63"
/dev/sda3 : start=      955904, size=   125829120, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=CBDC4617-6786-11E9-97EA-F82201FA303B
/dev/sda4 : start=   126785024, size=   251604480, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=CBDC4618-6786-11E9-97EA-F82201FA303B

content of d1.original.fstypes

cat d1.original.fstypes
/dev/sda1 ntfs
/dev/sda3 ntfs
/dev/sda4 ntfs

[Sebastian Roth]

@scosta There is one particular aspect of resizable images in FOG that you need to know about. So far we have not got to the point where we distinguish between partitions that need to be fixed in size and position (e.g. MBR boot partitions tat break booting if moved) and on the other hand partitions that should be fixed in size but don’t need to stay at the same location (e.g. SWAP partition).

The math of resizing is complex enough anyway and we have not had the time to go ahead and come up with a solution to that. For that reason FOG might try to resize your partitions 1, 3 and 4 but will not move those forward to make room and fit those on a smaller size disk.

While I have not had a closer look to the figures you posted I can imagine this being the problem here.

[Sebastian Roth]

@scosta Can you please try the following: Schedule another deploy task but tick the checkbox for debug this time. Boot up the client, hit ENTER twice and when you get to the shell run the command fog to start the task. Then step through it till you hit the error. You should get back to the shell after that. Run the following command, take a picture of the output and post here:

sgdisk -gl /images/W10_SEMSIS_MULTI_1.01/d1.mbr /dev/sda

[Quazz]

It looks like it failed to correctly record which partitions should be fixed.

FOG 1.5.6 should be better about that.

To fix it for this image specifically change d1.fixed_size_partitions from :2 to :1:2:3

That being said

I don’t believe that is your main issue here since the partition sizes don’t imply they are resizable anyway, but I’m not 100% sure if the system will or will not try to resize them under these conditions.

[scosta]

@Sebastian-Roth here is the test you asked me to run.

it seems the problem is what you mentioned about partition positioning and resizing. Is there any way to work around this?

@Quazz I’ll try your suggestion right now, be back in a minute! EDIT: i’m back, and the error persists…

P.S.: Sorry for taking so long to return the test results, busy day!

[Sebastian Roth]

@scosta I think we have all the information to try replicating the issue. Will do that over the weekend. If you are able to grab one more thing that might make it easier for us. Not sure if you got access over the weekend but if you easily can, get a copy of the binary d1.mbr, upload to your file share and post a link here.

[Sebastian Roth]

@scosta I can reproduce the error but only if I use a disk sized 60 GB or smaller. As soon as I use a disk around 120 GB (as you mentioned) I don’t have any issue deploying using the partition layout you posted.

Please re-run the debug deploy test on your VM and when it fails and you get to the shell please run fdisk -l /dev/sda, take a picture and post here.

It’s still possible I am missing something here. If so it’s probably something I can’t get from the partition layout text files. Then I may ask you to upload d1.mbr so I can test with that.

[scosta]

@Sebastian-Roth thanks for answering, and sorry for my absense, i was travelling to remote sites to implement more fogstorages

Here’s the output of fdisk as you requested:

here’s a link all files except the big img ones:
https://drive.google.com/drive/folders/1LLdjb_OsqiFrrOTesQDlxvrb0RF3hDUC?usp=sharing

[Sebastian Roth]

@scosta Thanks for uploading the files. I wouldn’t have had a chance without those. Turns out the partition start sector information is different in d1.partitions versus what we see in d1.mbr. I still have no idea how that can happen. It doesn’t match any logic I have come across so far. It’s not a 4k block size thing and I have no idea what’s going on - not yet.

But I have a quick fix for you. Get a good hex editor and a copy of your d1.mbr file (keep a copy of the original somewhere just in case). Open it and go down a few lines till you get to position 07A0. Should look like this:

00000780   A2 A0 D0 EB  E5 B9 33 44  87 C0 68 B6  B7 26 99 C7  ......3D..h..&..
00000790   18 46 DC CB  86 67 E9 11  97 EA F8 22  01 FA 30 3B  .F...g....."..0;
000007A0   00 02 D0 0E  00 00 00 00  F7 FC 9E 0F  00 00 00 00  ................

Simply change the hex numbers in the last line (07A0

00000780   A2 A0 D0 EB  E5 B9 33 44  87 C0 68 B6  B7 26 99 C7  ......3D..h..&..
00000790   18 46 DC CB  86 67 E9 11  97 EA F8 22  01 FA 30 3B  .F...g....."..0;
000007A0   00 96 8E 07  00 00 00 00  F7 90 5D 08  00 00 00 00  ..........].....

Save the file and put into your image directory on the FOG server as /images/W10_SEMSIS_MULTI_1.01/d1.mbr. Now try to deploy again. Should work.

I know this sounds really strange and I am fairly sure we’ll find out why this has happened. But for the time being I hope you can get this fixed as simple as described above to be able to deploy this image again.

[Sebastian Roth]

@scosta Thinking a bit more about it I am wondering if you did resize the partitions on your master at some point. I am not sure but possibly this could lead to the situation we see with different partition start information.

Please schedule a debug capture task and when you get to the shell run sfdisk -d /dev/sda and gdisk -l /dev/sda - take a picture and post here.

[scosta]

@Sebastian-Roth I’ll be running your requests asap, but I can antecipate you that the only manual alteration I tried before posting here was to adjust the d1.minimim.partitions sizes to match the d1.partitions, because using the d1.fixed_size_partitions was not forcing my system partition to 60Gb (/dev/sda3 and i need it to never be resized.)

Maybe the capture process have anything to do with this? What I did: Prepare the image using UEFI Bios, then switch to Legacy for the FOG Capture, then reverting back to UEFI. Same deal on the deploy: change to legacy to Fog Deploy, then reverting back to UEFI.

[Sebastian Roth]

@scosta Hey, any news on this?

We might have a similar issue here: https://forums.fogproject.org/topic/13302/error-trying-to-restore-gpt-partition-tables-exit-returned-code-4

Did you use mbr2gpt to convert your system from legacy to UEFI at some point in time?

[scosta]

@Sebastian-Roth yeah, we did… To solve the problem i did that HEX edition on the d1.mbr file you suggested and it fixed my issue. what still intrigues me is the fact that using d1.fixed_size_partitions, i.e. changing it to :2:3:4 is not ensuring those partitions to stay with fixed size as i needed. to make it happen, i had to make the sizes on both d1.partitions and d1.minium.partitions the same.

i’ll take a look on this other forum link you mentioned.

[Sebastian Roth]

@scosta Thanks for letting us know. Great to hear the hex edit foo worked for you as well. Very strange case indeed.

what still intrigues me is the fact that using d1.fixed_size_partitions, i.e. changing it to :2:3:4 is not ensuring those partitions to stay with fixed size as i needed.

Now about this. I don’t get what you say here. For the partition layout you posted above it doesn’t make much sense to me to set fixed to “:2:3:4” as this would force it to be a non-resizable deploy. Or is that what you are saying? You want it to be non-resizable? Why not use the right image type then. Sure the behavior might be a bit unexpected but this is kind of like saying I would like to have a car with automatic transmission but don’t want to have it change gear by itself. Well kind of, just kidding.

[scosta]

@Sebastian-Roth said in Error trying to restore GPT partition deploying an Image to Smaller Disk:

it doesn’t make much sense to me to set fixed to “:2:3:4”

indeed it does not make sense, because i confused myself haha =D
i meant to set fixed to “:1:2:3”, intending to keep the 3 first partitions fixed and the last one resizable as need. that also did not work.

i really don’t mind changing the files to achieve the desired result, but if fixed_size_partitions works, it would be a LOT more easier

[Sebastian Roth]

@scosta I know it’s been a long time but have not found enough time to work on this earlier. We now have a first version of init.xz capable of capturing and deploying a Windows 10 2004 UEFI/GPT install with recovery partition at the end. Would you give that a try.

Please be aware this is still in kind of a pre-release phase. Make sure you have a backup of your original image as well as the machine you pull the master from, just in case!!