Move partitions on GPT layouts - need people to test
-
UPDATE: This is now part the official inits!!
After several people noticed that Windows 10 2004 comes with a different partition layout that FOG is not able to shrink and deploy to a smaller size disk I started working on that code and adding the functionality to actually move the start sector of partitions. Up to now FOG never moved partitions because it would surely cause trouble as manipulating boot loader code in the MBR would also be needed and is way too error prone to do. With GPT partition layout on UEFI based systems partition start positions should not be “hardcoded” in bootloader code anymore.
Though this seems to work I would really like more people to test this before we actually add this to the official release!
Here are the steps I recommend when testing:
- Make sure you have a working backup copy of your host used to capture the image from! Just in case something goes wrong when capturing with the new init.
- Download the init file and put that in
/var/www/html/fog/service/ipxe/
on your FOG server. - Create a new image definition for this test.
- Decide which hosts you use for testing capture and deploy (the later having a smaller size disk to see if it works) with this new init, edit their hosts settings in the FOG web UI and set Host Init to
init-201114.xz
as well as the image definition you just created. - Schedule a capture task and pay attention to the boot process where it says
bzImage..ok
andinit-201114.xz..ok
to make sure it actually uses the new init file. - Schedule a deploy task for a machine with a smaller size disk and deploy to it. Again make sure it says
init-201114.xz..ok
when booting. - Please post here if you see issues as well as having success. Let us know what OS you have installed (Win 10 2004? Linux?) and the partition layout (contents of
/images/NEWIMAGENAME/d1.partitions
)!
References:
https://forums.fogproject.org/topic/14691/error-trying-to-restore-gpt-partition-when-deploying-image-to-smaller-drive-error-return-code-4
https://forums.fogproject.org/topic/13220/error-trying-to-restore-gpt-partition-deploying-an-image-to-smaller-disk
https://forums.fogproject.org/topic/14684/windows-8-1-corrupted-bcd
https://forums.fogproject.org/topic/15158/fog-failed-partition-issue
https://forums.fogproject.org/topic/15163/partitions-is-too-big-for-the-diskIssue fixed later on:
https://forums.fogproject.org/topic/15363/issue-with-single-disk-resizable -
@testers Anyone of you still active in the forums?
-
@sebastian-roth I hope to be building our 20H2 image the start of next year. So I can do some testing along with that for sure on this then.
-
@Sebastian-Roth I just re-read through this as I’m starting work on our 20H2 image.
We actually purposely make our base image from a smaller disk already of around 64 GB and it usually shrinks down to around 20 GB when captured. Then we deploy to things with larger drives.So just to be clear, I need to create the image from a new VM or physical machine that has a larger drive? i.e. expand the drive to something like 256 or 512 and then capture (where it will resize down to the usual 20ish GB or probably less for this test cause I’ll just install windows and that’s it) and then deploy to something with a 64 or 128 sized drive where it will resize.
Why are people creating images from things with drives larger than what they are deploying to?
Or is this testing a different type of image?
-
@jj-fullmer said in Move partition in GPT layout - need people to test:
So just to be clear, I need to create the image from a new VM or physical machine that has a larger drive? i.e. expand the drive to something like 256 or 512 and then capture (where it will resize down to the usual 20ish GB or probably less for this test cause I’ll just install windows and that’s it) and then deploy to something with a 64 or 128 sized drive where it will resize.
That sounds like a reasonable test to me!
Why are people creating images from things with drives larger than what they are deploying to?
Probably because they use some hardware machine they have as master but don’t think about the disk size making a difference.
-
@sebastian-roth I’m doing tests using just vms, so it’s possible that physical hardware could respond differently.
Here are my results.I created an image successfully from a 128 GB VM, it was 10 GB in size after being resized (all I did was enter audit mode after installing windows and then captured)
I deployed it to a VM with a 60GB drive and had no issues.
I did confirm that the new init was used at boot for both. There was a bunch of extra output that happened to fast to catch on the capture before the gparted screens showed up, but everything worked fine.
Here is my d1.partitions
label: gpt label-id: 68156B04-B4FE-40EF-96CC-747C33F75E54 device: /dev/nvme0n1 unit: sectors first-lba: 34 last-lba: 268435422 sector-size: 512 /dev/nvme0n1p1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=2C465DF5-10FD-4883-B323-423E3D10FB2E, name="EFI system partition", attrs="GUID:63" /dev/nvme0n1p2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=7AFE394E-47A0-43FE-A06E-7AFD7286E200, name="Microsoft reserved partition", attrs="GUID:63" /dev/nvme0n1p3 : start= 239616, size= 267168294, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=4A0FF43A-6E31-4AC9-8446-073E07C63AA8, name="Basic data partition" /dev/nvme0n1p4 : start= 267409408, size= 1021952, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=AEF97419-7B7D-4EC5-8C19-FF8C0217A965, attrs="RequiredPartition GUID:63"
Just to be safe, I will also test this in reverse with the init, Going to capture the 60 GB image and deploy it to the 128 GB VM, especially since that’s what I usually do.
Also, just for reference, here is the d1.partitions from our 1909 image. It looks like windows started putting the data partition at partition 3 instead of 4, kinda weird, that would make expanding disks for the main data drive more difficult, I wonder what the reasoning for that is.
[root@arrowfog Base-Stable]# cat d1.partitions label: gpt label-id: 68156B04-B4FE-40EF-96CC-747C33F75E54 device: /dev/sda unit: sectors first-lba: 34 last-lba: 134217694 sector-size: 512 /dev/sda1 : start= 2048, size= 1083392, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=75A55945-9352-4F30-A330-67878EC28FA1, name="Basic data partition", attrs="RequiredPartition GUID:63" /dev/sda2 : start= 1085440, size= 202752, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=D1336408-C5B3-446F-BE78-725587F7224D, name="EFI system partition", attrs="GUID:63" /dev/sda3 : start= 1288192, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=C9F8E03C-928A-4233-9837-49CE510E39B0, name="Microsoft reserved partition", attrs="GUID:63" /dev/sda4 : start= 1320960, size= 132894720, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=D7729B0D-A7B4-4E1F-A9F5-F06FAAAE6141, name="Basic data partition"
-
@jj-fullmer Thanks for testing! Looking fine so far I would say.
it was 10 GB in size after being resized
You mean the image size shown in the FOG web UI or when you booted it up to Windows again? The later would really be an issue. It should not stay shrinked.
Here is my d1.partitions
Could you post the contents of
d1.minimum.partitions
as well? -
@sebastian-roth I did not see this before I overwrote the test image, so I don’t have the d1.minimum.partitions, but I still can for the 60 GB image if you want that just to see what the init wrote. I can re-capture the larger vm after I finish the test for smaller to larger and get that for you, I’ll just post it tomorrow.
The 10 GB was the fog gui, everything expanded as it should in windows.
-
The smaller to larger worked fine as well.
here are all the d1.partition and d1.minimum.partitions I have (will update this post tomorrow with the recaptured first image) so it’s all in one spot1909 image
for reference of old windows partition table
This was NOT created with the new init, included this for comparison.d1.partitions
label: gpt label-id: 68156B04-B4FE-40EF-96CC-747C33F75E54 device: /dev/sda unit: sectors first-lba: 34 last-lba: 134217694 sector-size: 512 /dev/sda1 : start= 2048, size= 1083392, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=75A55945-9352-4F30-A330-67878EC28FA1, name="Basic data partition", attrs="RequiredPartition GUID:63" /dev/sda2 : start= 1085440, size= 202752, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=D1336408-C5B3-446F-BE78-725587F7224D, name="EFI system partition", attrs="GUID:63" /dev/sda3 : start= 1288192, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=C9F8E03C-928A-4233-9837-49CE510E39B0, name="Microsoft reserved partition", attrs="GUID:63" /dev/sda4 : start= 1320960, size= 132894720, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=D7729B0D-A7B4-4E1F-A9F5-F06FAAAE6141, name="Basic data partition"
d1.minimum.partitions
label: gpt label-id: 68156B04-B4FE-40EF-96CC-747C33F75E54 device: /dev/sda unit: sectors first-lba: 34 last-lba: 134217694 sector-size: 512 /dev/sda1 : start= 2048, size= 1083392, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=75A55945-9352-4F30-A330-67878EC28FA1, name="Basic data partition", attrs="RequiredPartition GUID:63" /dev/sda2 : start= 1085440, size= 202752, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=D1336408-C5B3-446F-BE78-725587F7224D, name="EFI system partition", attrs="GUID:63" /dev/sda3 : start= 1288192, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=C9F8E03C-928A-4233-9837-49CE510E39B0, name="Microsoft reserved partition", attrs="GUID:63" /dev/sda4 : start= 1320960, size= 45462454, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=D7729B0D-A7B4-4E1F-A9F5-F06FAAAE6141, name="Basic data partition"
20H2 smaller image
partitions of image from 60 GB VM
d1.partitions
label: gpt label-id: 68156B04-B4FE-40EF-96CC-747C33F75E54 device: /dev/nvme0n1 unit: sectors first-lba: 34 last-lba: 125829086 sector-size: 512 /dev/nvme0n1p1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=2C465DF5-10FD-4883-B323-423E3D10FB2E, name="EFI system partition", attrs="GUID:63" /dev/nvme0n1p2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=7AFE394E-47A0-43FE-A06E-7AFD7286E200, name="Microsoft reserved partition", attrs="GUID:63" /dev/nvme0n1p3 : start= 239616, size= 124567040, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=4A0FF43A-6E31-4AC9-8446-073E07C63AA8, name="Basic data partition" /dev/nvme0n1p4 : start= 124806656, size= 1021952, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=AEF97419-7B7D-4EC5-8C19-FF8C0217A965, attrs="RequiredPartition GUID:63"
d1.minimum.partitions
label: gpt label-id: 68156B04-B4FE-40EF-96CC-747C33F75E54 device: /dev/nvme0n1 unit: sectors first-lba: 34 last-lba: 125829086 sector-size: 512 /dev/nvme0n1p1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=2C465DF5-10FD-4883-B323-423E3D10FB2E, name="EFI system partition", attrs="GUID:63" /dev/nvme0n1p2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=7AFE394E-47A0-43FE-A06E-7AFD7286E200, name="Microsoft reserved partition", attrs="GUID:63" /dev/nvme0n1p3 : start= 239616, size= 21331078, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=4A0FF43A-6E31-4AC9-8446-073E07C63AA8, name="Basic data partition" /dev/nvme0n1p4 : start= 21571584, size= 1021952, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=AEF97419-7B7D-4EC5-8C19-FF8C0217A965, attrs="RequiredPartition GUID:63"
20H2 Larger image
Partitions of image from 128 GB VM
d1.partitions
label: gpt label-id: 68156B04-B4FE-40EF-96CC-747C33F75E54 device: /dev/nvme0n1 unit: sectors first-lba: 34 last-lba: 268435422 sector-size: 512 /dev/nvme0n1p1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=2C465DF5-10FD-4883-B323-423E3D10FB2E, name="EFI system partition", attrs="GUID:63" /dev/nvme0n1p2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=7AFE394E-47A0-43FE-A06E-7AFD7286E200, name="Microsoft reserved partition", attrs="GUID:63" /dev/nvme0n1p3 : start= 239616, size= 267173376, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=4A0FF43A-6E31-4AC9-8446-073E07C63AA8, name="Basic data partition" /dev/nvme0n1p4 : start= 267412992, size= 1021952, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=AEF97419-7B7D-4EC5-8C19-FF8C0217A965, attrs="RequiredPartition GUID:63"
d1.minimum.partitions
label: gpt label-id: 68156B04-B4FE-40EF-96CC-747C33F75E54 device: /dev/nvme0n1 unit: sectors first-lba: 34 last-lba: 268435422 sector-size: 512 /dev/nvme0n1p1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=2C465DF5-10FD-4883-B323-423E3D10FB2E, name="EFI system partition", attrs="GUID:63" /dev/nvme0n1p2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=7AFE394E-47A0-43FE-A06E-7AFD7286E200, name="Microsoft reserved partition", attrs="GUID:63" /dev/nvme0n1p3 : start= 239616, size= 21351120, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=4A0FF43A-6E31-4AC9-8446-073E07C63AA8, name="Basic data partition" /dev/nvme0n1p4 : start= 21592064, size= 1021952, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=AEF97419-7B7D-4EC5-8C19-FF8C0217A965, attrs="RequiredPartition GUID:63"
-
@sebastian-roth Do we also need to do some testing with this init on older versions of windows to be sure it doesn’t break anything there?
And Other OS’s too? Like maybe at least 1 linux distro -
@JJ-Fullmer Thanks for posting the other details as well. Looking all fine so far. We definitely want to do further tests.
Could you do one with Windows 7? Be sure to install in UEFI mode to get a GPT partition layout.
I can look into testing a Linux distro.
-
@sebastian-roth I’m working on a windows 8.1 w/update image (since windows 7 isn’t supported anymore)
Will let you know the results -
Windows 8.1
Image captured and deployed with new init
Did deploy test from larger 128 GB source VM and it had no problem deploying to 60 GB VMd1.partitions
label: gpt label-id: 68156B04-B4FE-40EF-96CC-747C33F75E54 device: /dev/nvme0n1 unit: sectors first-lba: 34 last-lba: 268435422 sector-size: 512 /dev/nvme0n1p1 : start= 2048, size= 614400, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=6D92B4A9-E0B4-4DED-8C4C-E271AA36B06F, name="Basic data partition", attrs="RequiredPartition GUID:63" /dev/nvme0n1p2 : start= 616448, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=2410B705-A7CB-4CE5-A393-8B3919C909D1, name="EFI system partition", attrs="GUID:63" /dev/nvme0n1p3 : start= 821248, size= 262144, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=58F556FE-7D3B-44EA-A846-C0848DB743CB, name="Microsoft reserved partition", attrs="GUID:63" /dev/nvme0n1p4 : start= 1083392, size= 267350016, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=364F4B4F-4D49-4955-A5EB-9CDE151EEBF7, name="Basic data partition"
d1.minimum.partitions
label: gpt label-id: 68156B04-B4FE-40EF-96CC-747C33F75E54 device: /dev/nvme0n1 unit: sectors first-lba: 34 last-lba: 268435422 sector-size: 512 /dev/nvme0n1p1 : start= 2048, size= 614400, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=6D92B4A9-E0B4-4DED-8C4C-E271AA36B06F, name="Basic data partition", attrs="RequiredPartition GUID:63" /dev/nvme0n1p2 : start= 616448, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=2410B705-A7CB-4CE5-A393-8B3919C909D1, name="EFI system partition", attrs="GUID:63" /dev/nvme0n1p3 : start= 821248, size= 262144, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=58F556FE-7D3B-44EA-A846-C0848DB743CB, name="Microsoft reserved partition", attrs="GUID:63" /dev/nvme0n1p4 : start= 1083392, size= 17806622, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=364F4B4F-4D49-4955-A5EB-9CDE151EEBF7, name="Basic data partition"
-
Hi,
i try to capture an 80Gb disk with the last version of Windows 10 but when the capture begin i have the following error :
https://cloud.fairy-family.fr/index.php/s/skJP4Tkrd2WSoxp
did i forget something ?
-
@btoffolon What version of FOG are you using? The init error your seeing would seem to indicate you need to change the init ramdisk size from 127000 to 275000. This can be done from FOG Configuration Page -> FOG Settings.
If this is already set to 275000 then it would seem the full init wasn’t downloaded.
-
@tom-elliott I use fog 1.5.9, the ramdisk size it’s at 275000 i will try to download again the init file and try again
thanks for the answer
-
@tom-elliott i download again the file but i have the same issue
-
@btoffolon That is kinda strange. I don’t think I have seen this error message before but it sounds like the init is corrupt or not fully loaded when booting. Though it doesn’t make sense to me as the original FOG 1.5.9 init works for you and other people tested this alpha stage init recently as well.
Please run those commands and post output here:
ls -al /var/www/html/fog/service/ipxe/init-201114.xz ls -al /var/www/fog/service/ipxe/init-201114.xz md5sum /var/www/html/fog/service/ipxe/init-201114.xz md5sum /var/www/fog/service/ipxe/init-201114.xz
-
@btoffolon Does this machine image fine with the original init?
You can also try a different bzImage. I’m using 5.618RT3 made by @george1421 found here https://forums.fogproject.org/topic/15017/lenovo-l13-yoga-maybe-the-usb-cable-is-bad-error-when-trying-to-register/18?_=1610731427661I may have forgotten that I was using a different bzImage kernel when I started testing for this…
-
@sebastian-roth @JJ-Fullmer Thanks for your help, for unknonwn reason the file was deleted well i put it a third time and its work perfectly, i capture a 500Gb HDD disk and deploy in 120Gb SSD.
d1.partitions
fog@fog:/images/Salle-Technologie$ cat d1.partitions label: gpt label-id: 79FE4766-EDD2-4DD0-826C-6DD816A2E7C6 device: /dev/sda unit: sectors first-lba: 34 last-lba: 976773134 sector-size: 512 /dev/sda1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=5217DE87-4A45-4AC4-A4DD-0C8EB1C5CF5B, name="EFI system partition", attrs="GUID:63" /dev/sda2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=15C5E73F-7855-4DA4-A464-4694B0E56628, name="Microsoft reserved partition", attrs="GUID:63" /dev/sda3 : start= 239616, size= 975486976, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=E130C384-3F46-4D8F-AD25-B85FDB0127CD, name="Basic data partition" /dev/sda4 : start= 975726592, size= 1046528, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=D3FA37D6-10A7-4303-976C-6D8F23C4258F, name="attrs=\x22RequiredPartition GUID:63", attrs="RequiredPartition GUID:63"
d1.minimum.partitions
fog@fog:/images/Salle-Technologie$ cat d1.minimum.partitions label: gpt label-id: 79FE4766-EDD2-4DD0-826C-6DD816A2E7C6 device: /dev/sda unit: sectors first-lba: 34 last-lba: 976773134 sector-size: 512 /dev/sda1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=5217DE87-4A45-4AC4-A4DD-0C8EB1C5CF5B, name="EFI system partition", attrs="GUID:63" /dev/sda2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=15C5E73F-7855-4DA4-A464-4694B0E56628, name="Microsoft reserved partition", attrs="GUID:63" /dev/sda3 : start= 239616, size= 72336566, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=E130C384-3F46-4D8F-AD25-B85FDB0127CD, name="Basic data partition" /dev/sda4 : start= 72577024, size= 1046528, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=D3FA37D6-10A7-4303-976C-6D8F23C4258F, name="attrs=\x22RequiredPartition GUID:63", attrs="RequiredPartition GUID:63"```