Issue with Single Disk (resizable)

[Eruthon]

Hi,

this week at work we wanted to upgrade 2x Acer Z3-605 AiO from 500GB HDDs to 240GB SSDs. Systems were installed with Win10 with GPT partitioning and UEFI boot.

We tried file ipxe.pxe with IPv4 Network Stack iPXE, which resulted in restart before even downloading bzImage.

We changed UEFI to Legacy (and undionly.kpxe) and used this with Single Disk (resizable); it managed to upload everything into the /images/dev folder, but the PC restarted and the task started again in infinite loop. This issue isn’t happening with other PCs, so I don’t think it’s FTP related.

We changed the Single Disk (resizable) to Single Disk - Multiple Partitions (not-resizable) and it managed to upload the image succesfully. Or at least it shows Image Size on Server now. We then resorted to copy with USB partition utilities, because there was not enough time to tackle with it (wasted 2 days with this).

FOG version 1.5.9 stable

So now just for explanation: what could be the problem here?

[Sebastian Roth]

@eruthon said in Acer Z3-605 AiO - Single Disk (resizable):

So now just for explanation: what could be the problem here?

There is no simple answer to this as you are describing various different issues. Let’s see if we can tackle those one by one. It might be wise to open new topics in the forums as we go along to focus on the different issues and not loose track. Depends on how dare you are so solve all of them.

this week at work we wanted to upgrade 2x Acer Z3-605 AiO from 500GB HDDs to 240GB SSDs. Systems were installed with Win10 with GPT partitioning and UEFI boot.

We tried file ipxe.pxe with IPv4 Network Stack iPXE, which resulted in restart before even downloading bzImage.

Have you tried other UEFI iPXE binaries like snponly.efi or snp.efi yet? The next step would be to compile new iPXE binaries from the latest source code to see if that fixes the PXE boot issue in UEFI mode. But we better look into the other this first.

We changed UEFI to Legacy (and undionly.kpxe) and used this with Single Disk (resizable);

That’s a good step to get around the UEFI PXE boot issue. FOG doesn’t care which mode it PXE boots in as long as it does it will start the task and do its job.

it managed to upload everything into the /images/dev folder, but the PC restarted and the task started again in infinite loop. This issue isn’t happening with other PCs, so I don’t think it’s FTP related.

Was the image moved to /images/#IMAGENAME#/ after the capture? I am fairly sure it was not. Sounds like FTP/disk space/access rights issue or the image capture ran into an issue and did not finish at all. Did you pay attention to the messages on screen when the capture was running? We need more information to be able to help with this.

We changed the Single Disk (resizable) to Single Disk - Multiple Partitions (not-resizable) and it managed to upload the image succesfully. Or at least it shows Image Size on Server now.

Better to check the files in /images/#IMAGENAME#/ on the server than relying on “Image Size on Server” information when in doubt if capture worked correctly. The non-resizable type won’t help you in this case because you can’t deploy it to the smaller size SSD.

FOG version 1.5.9 stable

This version of FOG is not yet able to capture and deploy an image to a smaller size disk if there is a recovery partition at the end of the partition layout. Find more information on this in the forums: https://forums.fogproject.org/topic/15025/move-partitions-on-gpt-layouts-need-people-to-test

[Eruthon]

@sebastian-roth
I found out it could be a problem with the new Windows 10 2004 and 20H2, where the Recovery partition causes problems discussed in this thread.

Today we tried to deploy an image with Windows 10 20H2 from 1TB NVMe to 240GB SSD, and it didn’t work until we shrank the Windows data partition to lower volume than ~240GB (we shrank to ~100GB) and moved the Recovery partition right after the data partition with Hiren’s Boot and AOMEI Partition Assistant, uploaded it and deployed it succesfully.

Tommorow I could try the new init file shared in the mentioned thread and post the result here.

[Sebastian Roth]

@eruthon said in Acer Z3-605 AiO - Single Disk (resizable):

Tommorow I could try the new init file shared in the mentioned thread and post the result here.

Find more details on this topic here: https://forums.fogproject.org/topic/15025/move-partitions-on-gpt-layouts-need-people-to-test

Better use the official developer inits as they were updated to include this fix some weeks ago.
http://fogproject.org/inits/init.xz and http://fogproject.org/inits/init_32.xz

[Eruthon]

@sebastian-roth
I tried the new init.xz (named it init-new.xz) and created new image, I used the Host Init option on both the source PC and destination PC; capturing went smoothly, but deploying to the smaller disk didn’t, it’s showing the same error with Partition 4:

Here’s the d1.minimum.partitions file:

label: gpt
label-id: D7496BEB-B8A7-4BDA-84D1-A924BDEB9789
device: /dev/nvme0n1
unit: sectors
first-lba: 34
last-lba: 1953525134
sector-size: 512

/dev/nvme0n1p1 : start=        2048, size=      204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=25DAF354-4243-415E-9163-8AA61F158BBC, name="EFI system partition", attrs="GUID:63"
/dev/nvme0n1p2 : start=      206848, size=       32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=3C51C90A-87D5-4C14-9D3E-378D17FF0882, name="Microsoft reserved partition", attrs="GUID:63"
/dev/nvme0n1p3 : start=      239616, size=    61921776, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=EE6CC19C-A52C-4909-B57D-E2915E54BB55, name="Basic data partition"
/dev/nvme0n1p4 : start=  1952491908, size=      884386, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=17790088-AF7B-4D4B-B155-8C7E9DAFE516

[Sebastian Roth]

@eruthon said in Acer Z3-605 AiO - Single Disk (resizable):

...
/dev/nvme0n1p3 : start=      239616, size=    61921776, ...
/dev/nvme0n1p4 : start=  1952491908, size=      884386, ...

Interesting. See the gap between partition three and the start of partition 4. It was not able to move partition 4 forward when capturing.

Did you pay close attention to it booting up to init-new.xz when capturing?

If the answer is yes, then I ask you to do another capture. But this time set ismajordebug=1 as Kernel Parameter for the host to be captured and enable the checkbox for debug (just before you click the “create task” button in the web UI there is a checkbox). Then boot it up to the task and you end up in a command shell. There you type fog as command to start. Then step through by pressing enter.

Now pay attention to the part where it says Moving /dev/nvme0n1p4 forward to close gap ... and take pictures of all the output you get on screen whenever you hit enter again. Post pictures here.

[Eruthon]

@sebastian-roth
Changed the host properties:

Scheduled as debug task:

iPXE initializing:

Variable dump from FOG:

FOG prep:

Moving of nvme0n1p4; the partition 4 in the “after” table has corrected start address:

Partclone:

Also the d1.minimum.partitions looks correct:

label: gpt
label-id: D7496BEB-B8A7-4BDA-84D1-A924BDEB9789
device: /dev/nvme0n1
unit: sectors
first-lba: 34
last-lba: 1953525134
sector-size: 512

/dev/nvme0n1p1 : start=        2048, size=      204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=25DAF354-4243-415E-9163-8AA61F158BBC, name="EFI system partition", attrs="GUID:63"
/dev/nvme0n1p2 : start=      206848, size=       32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=3C51C90A-87D5-4C14-9D3E-378D17FF0882, name="Microsoft reserved partition", attrs="GUID:63"
/dev/nvme0n1p3 : start=      239616, size=    61985414, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=EE6CC19C-A52C-4909-B57D-E2915E54BB55, name="Basic data partition"
/dev/nvme0n1p4 : start=    62225408, size=      884224, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=17790088-AF7B-4D4B-B155-8C7E9DAFE516

Edit:
Now I’m triying to deploy it to the smaller SSD and it works so far… interesting that I didn’t change anything since when it wasn’t working.

Edit:
It succesfully deployed the image to the SSD and Windows booted without problems.

[Sebastian Roth]

@Eruthon Awesome documentation! If everyone in the forums would post that many details we wouldn’t have to ask that many questions. Well done.

Interesting that it didn’t work on the first try. In the other topic we also had someone report that the last partition was not moved forward on the first try. As we don’t have any details on it I just can’t help it. So far it always worked in my tests on the first go. But I can’t proof there is no issue in the scripts. So please keep your eyes open and let us know if it happens again.

The issue occurs on capture already. Deploy can’t fix it if the partition is not moved forward and you see this in d1.minimum.partitions right after the capture.

Edit: Just noticed that FOG detected only one NTFS partition in your setup. Shouldn’t the recovery partitions be NTFS as well?!

[Eruthon]

@sebastian-roth
Thank you for the appreciation!

Also maybe I just overlooked something, maybe edited wrong host, as I had multiple tabs and didn’t keep track of it. Otherwise I don’t see any other reason for it not working on the first try.

[Eruthon]

After some time I tried to deploy new Win10 21H1 image with some pre-installed software again with UEFI and GPT… also using the new init… sadly it didn’t capture the partitions correctly again… lost my nerve, so didn’t try the debug mode, but will try next week

[Sebastian Roth]

@Eruthon Before you capture again, please get us a copy of the text files /images/IMAGENAME/d1.minimum.partitions and post the contents here in the forums.

If you have enough disk space on your FOG server you might even leave that image untouched and create a new image definition for the next capture/deploy test. It might be handy we have this “not working” one for debugging.

[Eruthon]

@sebastian-roth In the heat of the moment I tried to edit the data partition size in that file to try it that way; I didn’t think it would work, but tried it as last resort. The issue persisted, also the log during deploying showed the same amount of blocks that were problematic because of smaller disk size.

Sadly I didn’t back up the original file, but before the change I saw that the d1.partitions had the same values as the d1.minimum.partitions, so I copied the values from d1.partitions back after trying the mentioned experiment.

d1.minimum.partition for the new 21H1 image:

label: gpt
label-id: D7496BEB-B8A7-4BDA-84D1-A924BDEB9789
device: /dev/nvme0n1
unit: sectors
first-lba: 34
last-lba: 1953525134
sector-size: 512

/dev/nvme0n1p1 : start=        2048, size=      204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=25DAF354-4243-415E-9163-8AA61F158BBC, name="EFI system partition", attrs="GUID:63"
/dev/nvme0n1p2 : start=      206848, size=       32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=3C51C90A-87D5-4C14-9D3E-378D17FF0882, name="Microsoft reserved partition", attrs="GUID:63"
/dev/nvme0n1p3 : start=      239616, size=    77211472, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=EE6CC19C-A52C-4909-B57D-E2915E54BB55, name="Basic data partition"
/dev/nvme0n1p4 : start=  1952640000, size=      884224, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=17790088-AF7B-4D4B-B155-8C7E9DAFE516

The starting lba of the partition 4 is the same as in the posts of the 20H2 image you can see in my earlier posts. My point is I don’t see any differences between the working 20H2 image and the new 21H1 image, except for the size of the data partitions which is, ofc, expected.

Also I wonder if the name of the issue here could be changed as it’s not longer a suitable name for the problem here. Could be changed to sth regarding the gpt partitions, so it wouldn’t sound like a problem with the PC model mentioned in the title.

[Sebastian Roth]

@Eruthon No worries, I think that’s telling us that sometimes the move of the partition is not working. We see that size of nvme0n1p3 is way smaller than the start of nvme0n1p4. In that case the new init should notice the gap between those two partitions and move nvme0n1p4 forward for capturing (and move it back to it’s original position afterwards).

Though without further information it’s kind of impossible to figure out why this happens. It never happened to me when I came up with this and tested on my systems. So I can’t replicate the issue as of now.

Possibly this issue only can happen when you don’t run in debug mode anyway. Kind of a timing issue. We have had such a thing before and it’s really nasty to find and fix. But hey, I think there is no other change than trying major debug over and over on your system and see if it happens at some point.

PS: Changed the title.

[Sebastian Roth]

@Eruthon Good news! I figured out why this would fail and fixed the issue. Please re-download the inits (http://fogproject.org/inits/init.xz and http://fogproject.org/inits/init_32.xz), put in /var/www/html/fog/service/ipxe/ and capture again (no debug mode).

[Eruthon]

@sebastian-roth, thank you, files downloaded and ready, will report back with results tommorow!

[Eruthon]

@sebastian-roth, it’s working! Managed to deploy the image from 1TB to 240GB, hopefully it will work for others, too. Thank you verymuch for your time! Should I post some files here for reference?

[Sebastian Roth]

@Eruthon Thanks for the quick feedback. Should be all fine now I think. No need to post partition files as long as it works for you now.