FOG deploy: partitions 4 and 5 too big for disk

[madeyem]

Hi everyone,

I’m on FOG 1.5.9.111 on Ubuntu Server 20.04.

bzImage Version: 5.10.56
bzImage32 Version: 5.10.56

I am trying to deploy a Windows 10 20H2 UEFI image, which I took from an SSD machine on FOG 1.5.7, to a HDD machine, both SSD and HDD are 500 GB.

I am having the same problem as described here:

https://forums.fogproject.org/topic/14691/error-trying-to-restore-gpt-partition-when-deploying-image-to-smaller-drive-error-return-code-4?lang=de&page=1&sort=oldest_to_newest

What I tried so far:

1. Update from 1.5.9 to 1.5.9.111 to get the new inits.
2. Create new image definition in UI.
3. Re-Capture image from SSD machine.
4. Deploy on HDD machine.

But I still get the error (“partition 4 (and 5) too big for disk”).

d1.minimum.partitions:

label: gpt
label-id: 8C0CB60E-93F5-4C14-B633-3852928F826C
device: /dev/nvme0n1
unit: sectors
first-lba: 34
last-lba: 1000215182
sector-size: 512

/dev/nvme0n1p1 : start=        2048, size=     1433600, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=2AFFEC30-5F77-4C6B-BA7C-902A01A971E1, name="EFI system partition", attrs="GUID:63"

/dev/nvme0n1p2 : start=     1435648, size=      262144, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=E8336B4A-8C8F-48F5-85DE-E6237B7A372F, name="Microsoft reserved partition", attrs="GUID:63"

/dev/nvme0n1p3 : start=     1697792, size=   995418624, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=38131294-BD57-43F5-ACE4-2E1413614C9A, name="Basic data partition"

/dev/nvme0n1p4 : start=   997116416, size=     1071104, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=3F15511F-848F-4404-B079-FB3BE8D4EFB8, name="attrs=\x22RequiredPartition GUID:63"

/dev/nvme0n1p5 : start=   998187520, size=     2027520, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=84072955-EDFD-4E05-84E6-C3A6C50CDB30, name="attrs=\x22RequiredPartition GUID:63"

d1.partitions:

label: gpt
label-id: 8C0CB60E-93F5-4C14-B633-3852928F826C
device: /dev/nvme0n1
unit: sectors
first-lba: 34
last-lba: 1000215182
sector-size: 512

/dev/nvme0n1p1 : start=        2048, size=     1433600, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=2AFFEC30-5F77-4C6B-BA7C-902A01A971E1, name="EFI system partition", attrs="GUID:63"

/dev/nvme0n1p2 : start=     1435648, size=      262144, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=E8336B4A-8C8F-48F5-85DE-E6237B7A372F, name="Microsoft reserved partition", attrs="GUID:63"

/dev/nvme0n1p3 : start=     1697792, size=   995418624, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=38131294-BD57-43F5-ACE4-2E1413614C9A, name="Basic data partition"

/dev/nvme0n1p4 : start=   997116416, size=     1071104, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=3F15511F-848F-4404-B079-FB3BE8D4EFB8, name="attrs=\x22RequiredPartition GUID:63"

/dev/nvme0n1p5 : start=   998187520, size=     2027520, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=84072955-EDFD-4E05-84E6-C3A6C50CDB30, name="attrs=\x22RequiredPartition GUID:63"

I’ve already read other threads regarding this problem, but I must be missing something. Any help would be appreciated.

[Sebastian Roth]

@madeyem You are definitely on the right track with updating to the latest FOG version (1.5.9.111 as of now).

Taking a closer look at the partition table I see that there does not seem to be any free space to work with. Partition one starts at sector 2048 and is 1433600 sectors long.
2048 + 1433600 = 1435648
1435648 + 262144 = 1697792
1697792 + 995418624 = 997116416
997116416 + 1071104 = 998187520
998187520 + 2027520 = 1000215040

Usually FOG would shrink one of the partitions to make room and then move partitions forward to fit all of them on a smaller size disk.

both SSD and HDD are 500 GB

In the world of hard drives the size given by vendors is just a very rough “estimate”. The figures we see in the partition layout tell us this disk is actually 476.9 GB in size (1000215182 * 512 byte sector size / 1024 / 1024 / 1024).

To find out the sector count of the other disk you can run blockdev --getsz /dev/sda in a FOG debug session on that host.

Main question is, why is none of the five partitions being shrunk? I can imagine you have the same issue that we’ve seen in this topic: https://forums.fogproject.org/topic/15215/deploy-issue-after-copying-image-to-an-other-fog-server

[madeyem]

@sebastian-roth I will try the solution from the thread you referred to and report back.

Another question:

After updating to 1.5.9.111, do I have to tell FOG to use a special init or so or is the fix included in the default init? So far I did not try any different settings after updating to 1.5.9.111.

[madeyem]

@sebastian-roth After applying the steps described here for my partitions 4 and 5, it’s still not working. After deploying in debug mode, I get this output:

[Sebastian Roth]

@madeyem said in FOG deploy: partitions 4 and 5 too big for disk:

After updating to 1.5.9.111, do I have to tell FOG to use a special init or so or is the fix included in the default init? So far I did not try any different settings after updating to 1.5.9.111.

When you update to the latest dev-branch version it will pull the latest inits from github and use those straight away. no extra step needed.

After applying the steps described here for my partitions 4 and 5, it’s still not working. After deploying in debug mode, I get this output

I might have been on the wrong track with this. Looking at this again now I am wondering why nvme0n1p3 (third partition) is that big and obviously not being shrunk on capture!??!

Please schedule a debug capture task on the source host/drive and take pictures of the whole capture process so we see what’s actually happening.

[madeyem]

@sebastian-roth said in FOG deploy: partitions 4 and 5 too big for disk:

Please schedule a debug capture task on the source host/drive and take pictures of the whole capture process so we see what’s actually happening.

There you go:

https://imgur.com/a/dNq1AWF

(Imgur screwed up the right order of the pictures)

[Sebastian Roth]

@madeyem Sorry for the delay. Great picture show! You definitely use the latest FOS init (07.08.2021). From what I can see the partitions 3, 4 and 5 should be shrunk correctly. Can you please post the contents of d1.minimum.partitions of this newly captured image again?

Edit: Ok, no need to post the d1.minimum.partitions again as I am fairly sure it looks exactly the same as last time. Looked at the pictures again and saw this:

...
 * Moving Partitions on disk /dev/nvme0n1
 * Press [Enter] key to continue

 * Saving shrunken partition table
 * Press [Enter] key to continue
...

But if there was space between the partitions after resize/shrinking then we should see the message * Moving /dev/nvme0n1pX forward to close gap between end of /dev/nvme0n1pY and start of /dev/nvme0n1pX. at least once! code ref

For example partition 3 is 995418624 sectors, which amounts for ~ 475 GB partition size befor shrinking. In the pictures we see it say 75542334k to be the possible smallest size - 72 GB. So there is definitely a lot of room to shrink this filesystem and partition. In the pictures there is no message that makes me think the shrinking fails at any point.

So as a next step you want to add isdebug=yes ismajordebug=1 to the host settings (in Kernel Argument). Capture once again and take another series of pictures.

[madeyem]

@sebastian-roth said in FOG deploy: partitions 4 and 5 too big for disk:

So as a next step you want to add isdebug=yes ismajordebug=1 to the host settings (in Kernel Argument). Capture once again and take another series of pictures.

Is there a way to troubleshoot this without having to re-capture? The PC I used for re-capturing is now being used by a new colleague. I am currently out of PCs (new ones are ordered, but have long delivery times).

I currently only have older PCs here, but can’t deploy said image to them as described in my opening post.

[Sebastian Roth]

@madeyem Don’t you have a so called master machine that you use to build your “golden setup” on to capture an image from (the same machine every time)?

Is there a way to troubleshoot this without having to re-capture?

Not that I know of. We are missing some information here. No idea why this fails in your specific case. Does not seem to be a general issue.

[madeyem]

@sebastian-roth said in FOG deploy: partitions 4 and 5 too big for disk:

@madeyem Don’t you have a so called master machine that you use to build your “golden setup” on to capture an image from (the same machine every time)?

Unfortunately not.

Not that I know of. We are missing some information here. No idea why this fails in your specific case. Does not seem to be a general issue.

I can say, that I did not have any problems under FOG 1.5.7, which I used for a long time. I will see, what I can do.

[Sebastian Roth]

@madeyem said in FOG deploy: partitions 4 and 5 too big for disk:

I can say, that I did not have any problems under FOG 1.5.7, which I used for a long time. I will see, what I can do.

I can say that FOG 1.5.7 would not have been able to shrink this layout either I am pretty sure!

[madeyem]

@sebastian-roth said in FOG deploy: partitions 4 and 5 too big for disk:

@madeyem said in FOG deploy: partitions 4 and 5 too big for disk:

I can say, that I did not have any problems under FOG 1.5.7, which I used for a long time. I will see, what I can do.

I can say that FOG 1.5.7 would not have been able to shrink this layout either I am pretty sure!

ok

[Sebastian Roth]

@madeyem Ok, I had another look at this and I have to say that my initial thought was right. Your issue is caused by the same problem discussed in the other forum topic mentioned before.

I can say that FOG 1.5.7 would not have been able to shrink this layout either I am pretty sure!

Proofed myself wrong!

I will look into fixing what is causing this in the inits. Will let you know as soon as I have it ready for testing.

[madeyem]

@sebastian-roth said in FOG deploy: partitions 4 and 5 too big for disk:

@madeyem Ok, I had another look at this and I have to say that my initial thought was right. Your issue is caused by the same problem discussed in the other forum topic mentioned before.

I can say that FOG 1.5.7 would not have been able to shrink this layout either I am pretty sure!

Proofed myself wrong!

I will look into fixing what is causing this in the inits. Will let you know as soon as I have it ready for testing.

Ok, thank you!

[Sebastian Roth]

@madeyem Please download this FOS init and try capture and deploy again. The issue should be fixed.

[madeyem]

@sebastian-roth said in FOG deploy: partitions 4 and 5 too big for disk:

@madeyem Please download this FOS init and try capture or deploy again. The issue should be fixed.

I’m currently on vacation for two weeks. I will report back, as soon as I’m back in the office and have tested it. Thanks!

[madeyem]

@sebastian-roth said in FOG deploy: partitions 4 and 5 too big for disk:

@madeyem Please download this FOS init and try capture or deploy again. The issue should be fixed.

I replaced the default init.xz with the one you linked and tried to deploy, but unfortunately I still get the same error message. The correct init version is shown (20211009).

I can try to re-capture an image as soon as I have one of the SSD machines at my disposal (or when the new PCs I ordered finally arrive).

[madeyem]

FYI:
Just for the fun of it, I tried to deploy an Windows 7 image from 2018, which worked perfectly fine, but all my Windows 10 images fail.

[Sebastian Roth]

@madeyem said in FOG deploy: partitions 4 and 5 too big for disk:

I replaced the default init.xz with the one you linked and tried to deploy, but unfortunately I still get the same error message.

Sure thing. As I said in my last post you should capture and deploy. In your case the already captured image doesn’t have the proper shrunken partition layout and therefore can’t be deployed to a smaller size disk I suppose. If you try deploy those Win 10 images to a large disk, do they fail just the same way? If so I ask you to take a picture of the error on screen and post that here.

[madeyem]

@sebastian-roth said in FOG deploy: partitions 4 and 5 too big for disk:

Sure thing. As I said in my last post you should capture and deploy. In your case the already captured image doesn’t have the proper shrunken partition layout and therefore can’t be deployed to a smaller size disk I suppose. If you try deploy those Win 10 images to a large disk, do they fail just the same way? If so I ask you to take a picture of the error on screen and post that here.

Ok, I think I might get my hands on one of the SSD machines later this (or next) week. I will then re-capture, deploy and report back.