Surface Go 4 incompatible
-
Should also mention I’m using the latest available linux kernel version for the test 6.6.1
-
Found that enabling just the PCI option
PCI bus based UFS Controller support
is all that is needed for the disk to show in FOS
On my first attempt to image though it gave errors regarding partition overlap and partition size. The disk shows in fog as 119 GB, my win 10 image (single disk resizable) is under 22 GB. Not sure how it isn’t expanding into the disk, there may be something else special needed.
Trying a different image. -
@JJ-Fullmer No difference with a different image.
This is the error message
Here is the ls -l of that image
ls -l /images/Base-Stable/ total 11098480 -rwxrwxr-x 1 fogproject root 6 Oct 12 07:34 d1.fixed_size_partitions -rwxrwxr-x 1 fogproject root 1048576 Oct 12 07:35 d1.mbr -rwxrwxr-x 1 fogproject root 872 Oct 12 07:35 d1.minimum.partitions -rwxrwxr-x 1 fogproject root 20 Oct 12 07:34 d1.original.fstypes -rwxrwxr-x 1 fogproject root 0 Oct 12 07:34 d1.original.swapuuids -rwxrwxr-x 1 fogproject root 10722570 Oct 12 07:35 d1p1.img -rwxrwxr-x 1 fogproject root 754 Oct 12 07:35 d1p2.img -rwxrwxr-x 1 fogproject root 11352884934 Oct 12 07:40 d1p3.img -rwxrwxr-x 1 fogproject root 153330 Oct 12 07:40 d1p4.img -rwxrwxr-x 1 fogproject root 872 Oct 12 07:34 d1.partitions -rwxrwxr-x 1 fogproject root 872 Oct 12 07:35 d1.shrunken.partitions
And the contents of d1.partitions
cat /images/Base-Stable/d1.partitions label: gpt label-id: 745A6813-735A-4940-840D-28D0E805850A device: /dev/nvme0n1 unit: sectors first-lba: 34 last-lba: 83886046 sector-size: 512 /dev/nvme0n1p1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=CFB2D745-2975-480D-BD56-EA18B65E03C5, name="EFI system partition", attrs="GUID:63" /dev/nvme0n1p2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=57426B89-9DEE-4BDF-9CBF-E9452D985FD5, name="Microsoft reserved partition", attrs="GUID:63" /dev/nvme0n1p3 : start= 239616, size= 57417925, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=857B1626-9DC1-44FD-8120-2AB922BBBD85, name="Basic data partition" /dev/nvme0n1p4 : start= 82761728, size= 1120256, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=DE086091-5C6A-4D63-89E2-579581B9BD63, attrs="RequiredPartition GUID:63"
Maybe it’s because it’s captured as /dev/nvmeXX instead of /dev/sdaXX ? Just brainstorming there.
Doing an update to the latest dev version just to be sure this isn’t fixed in a recent patch.
-
Made a pull request for including the ufs in the kernel config
https://github.com/FOGProject/fos/pull/78Probably should get it fully working before doing that pull though
@SEBASTIAN-ROTH any thoughts?
-
@JJ-Fullmer I’m currently working on new kernels and inits. Having issues with building partclone for the inits, but I’ll hopefully have some time this weekend to get it done.
I will include the ufs changes for the kernels. -
So more findings
Did a test of a different image that is based on
/dev/sda
instead of/dev/nvmeXnX
That image deployed with the modified kernel just fine (minus booting to the OS, but that was a windows server image, which I didn’t expect to work on a surface go).
But wait there’s more!
As I started playing around in a debug session I found that the UFS disk wants to have a sector size of 4096. This is the dump after manually creating the partitions to match the d1.minimum.partitions of the image
sfdisk --dump /dev/sda label: gpt label-id: 745A6813-735A-4940-840D-28D0E805850A device: /dev/sda unit: sectors first-lba: 6 last-lba: 31246330 sector-size: 4096 /dev/sda1 : start= 256, size= 25600, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=CFB2D745-2975-480D-BD56-EA18B65E03C5, name="EFI system partition", attrs="GUID:63" /dev/sda2 : start= 25856, size= 4096, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=57426B89-9DEE-4BDF-9CBF-E9452D985FD5, name="Microsoft reserved partition", attrs="GUID:63" /dev/sda3 : start= 29952, size= 7340032, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=857B1626-9DC1-44FD-8120-2AB922BBBD85, name="Basic data partition" /dev/sda4 : start= 7369984, size= 140032, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=DE086091-5C6A-4D63-89E2-579581B9BD63, attrs="RequiredPartition GUID:63"
Whereas the header of d1.minimum.partitions in my normal base image is 512, though it seems like it might actually be 2048 based on math of partitionSectors/sector-size = size in MB (but I might be wrong about that formula, it’s been a minute since I’ve had to deal with sectors).
This is the contents of my d1.minimum.partitions from the image I want to deploy
label: gpt label-id: 745A6813-735A-4940-840D-28D0E805850A device: /dev/nvme0n1 unit: sectors first-lba: 34 last-lba: 83886046 sector-size: 512 /dev/nvme0n1p1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=CFB2D745-2975-480D-BD56-EA18B65E03C5, name="EFI system partition", attrs="GUID:63" /dev/nvme0n1p2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=57426B89-9DEE-4BDF-9CBF-E9452D985FD5, name="Microsoft reserved partition", attrs="GUID:63" /dev/nvme0n1p3 : start= 239616, size= 57417925, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=857B1626-9DC1-44FD-8120-2AB922BBBD85, name="Basic data partition" /dev/nvme0n1p4 : start= 82761728, size= 1120256, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=DE086091-5C6A-4D63-89E2-579581B9BD63, attrs="RequiredPartition GUID:63"
I tried what @Sebastian-Roth mentions in another post here https://forums.fogproject.org/topic/15608/imaging-from-large-drive-to-small-drive/4?=1700488459744 and what @Tom-Elliott mentions here https://forums.fogproject.org/topic/13004/error-trying-to-restore-gpt-partition-tables-on-multiple-dell-machines/13?=1700488480736
When I try to restore from the original file I get an error saying the
last-lba
is invalid/too high for the disk.What’s weird, is I can use that syntax to restore the server based image, and that min parts file also has a higher lba, but it makes a little sense as the final sector is below the drive’s “31246330” last lba.
cat ../SVR-Base-Stable/d1.minimum.partitions label: gpt label-id: 0B6865A1-3053-4F45-9371-B73F033C7395 device: /dev/sda unit: sectors first-lba: 34 last-lba: 188743646 sector-size: 512 /dev/sda1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=3DF3DEBB-C856-4CC2-9263-EAF0AABF2DDF, name="EFI system partition", attrs="GUID:63" /dev/sda2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=C9738054-91F0-4B3A-AFFA-A9C9ABCC9663, name="Microsoft reserved partition", attrs="GUID:63" /dev/sda3 : start= 239616, size= 27175378, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=42364778-3F9E-4421-88C1-698974428C14, name="Basic data partition", attrs="GUID:63" /dev/sda4 : start= 27415040, size= 1269760, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=CA25B771-91EC-40A3-B200-1360371A0AD8, attrs="RequiredPartition GUID:63"
So I made a backup of my original image mbr and patition files and edited the d1.minimum.partitions to look like this
label: gpt label-id: 745A6813-735A-4940-840D-28D0E805850A device: /dev/nvme0n1 unit: sectors first-lba: 6 last-lba: 31246330 sector-size: 4096 /dev/nvme0n1p1 : start= 256, size= 25600, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=CFB2D745-2975-480D-BD56-EA18B65E03C5, name="EFI system partition", attrs="GUID:63" /dev/nvme0n1p2 : start= 25856, size= 4096, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=57426B89-9DEE-4BDF-9CBF-E9452D985FD5, name="Microsoft reserved partition", attrs="GUID:63" /dev/nvme0n1p3 : start= 29952, size= 7340032, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=857B1626-9DC1-44FD-8120-2AB922BBBD85, name="Basic data partition" /dev/nvme0n1p4 : start= 7369984, size= 140032, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=DE086091-5C6A-4D63-89E2-579581B9BD63, attrs="RequiredPartition GUID:63"
Making it match the dump after making the partitions by MB and GB size manually.
After editing it this command worked without error
sfdisk /dev/sda < /images/Base-Stable/d1.minimum.partitions
So I made a new version of the d1.mbr
sgdisk -b /mnt/d1.mbr /dev/sda
And popped that on the server to test deploying with that. ON this test, I’m not editing the d1.partitions or d1.shrunken.partitions which may make it so it doesn’t expand properly, but we’ll see how it goes…
It got past the previous errors and is now imaging! woo hoo!
How to fix this permanently though, that’s a different question. -
The image did complete but it did not boot into Windows. Looks like I need to embed the storage controller driver for UFS.
Or it could be a 512b vs 4k block disk type issue.
-
@Sebastian-Roth or @Tom-Elliott any chance you have some time to help me out with this issue?
It’s looking like its a 4k block issue.
I was trying to find a formulaic way to change the partitions files to line up to the 4k layout, like dividing by 8 and some other tweaking as needed but found it more complicated than expected.Any ideas?
-
It’s looking like I have to find a way to either capture the image or a separate image from a device that defaults to 4k logical sectors or a way to force the device to change sector size to 512 when deploying a 512e based image
I tried
hdparm
but got this errorhdparm --set-sector-size 512 --please-destroy-my-drive /dev/sda /dev/sda: READ_LOG_EXT(SECTOR_CONFIGURATION) failed: No such file or directory
I also tried capturing the image using the new kernel on the off chance that would make a difference, but no good.
I guess I will have to try making an image from a 4k native device since it’s not being changed?
-
@JJ-Fullmer I captured an image from the surface go 4 with the new kernel that has UFS support and I was able to deploy that image with its 4k block based partition table without issue.
Of course that image can now not be deployed on 512e block based disks.If I ever get the time, I may try testing manipulation of the parition blocks and using an efi tool in a fog post install script to make it so an image can be deployed to either but its probably unlikely to be possible.
For reference, here’s what the 4k block based partitions look like
d1.partitions
label: gpt label-id: DEECB42F-8F01-49B9-85BE-8398A932E4E5 device: /dev/sda unit: sectors first-lba: 6 last-lba: 31246330 sector-size: 4096 /dev/sda1 : start= 256, size= 76800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=FE5BA595-3498-4D4A-B741-9EB5D84C1EA5, name="EFI system partition", attrs="GUID:63" /dev/sda2 : start= 77056, size= 4096, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=752C5B50-96B7-4130-9789-9A02B05F8A75, name="Microsoft reserved partition", attrs="GUID:63" /dev/sda3 : start= 81152, size= 31024716, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=C088525B-56B9-4D7E-A040-1C8F5A782ADC, name="Basic data partition" /dev/sda4 : start= 31106048, size= 139776, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=E2B6791E-DC55-4039-A658-9007B5749FEE, attrs="RequiredPartition GUID:63"
d1.minimum.partitions
label: gpt label-id: DEECB42F-8F01-49B9-85BE-8398A932E4E5 device: /dev/sda unit: sectors first-lba: 6 last-lba: 31246330 sector-size: 4096 /dev/sda1 : start= 256, size= 76800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=FE5BA595-3498-4D4A-B741-9EB5D84C1EA5, name="EFI system partition", attrs="GUID:63" /dev/sda2 : start= 77056, size= 4096, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=752C5B50-96B7-4130-9789-9A02B05F8A75, name="Microsoft reserved partition", attrs="GUID:63" /dev/sda3 : start= 81152, size= 31024716, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=C088525B-56B9-4D7E-A040-1C8F5A782ADC, name="Basic data partition" /dev/sda4 : start= 31106048, size= 139776, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=E2B6791E-DC55-4039-A658-9007B5749FEE, attrs="RequiredPartition GUID:63"
Something odd is it doesn’t look like the main OS partition (sda3) shrunk, but it for sure did as it is only 10.56 GB on the server (compressed size) and deploys around 20 GB on the client, same size as the VM based 512e image.
-
@JJ-Fullmer I just took another look as I’ve been just maintaining a separate 4kn image and realized that the disk isn’t expanding after imaging.
I’ll do a debug session tomorrow and report back. -
Did a debug session and
ntfsresize
with -c to check and --info shows partition 3 as resizable but it’s not being resized after imaging.Running the image in deploy and seeing what it says after imaging and will see if there are any errors.
I fear this is going to be a 4kn drive alignment/resize issue.
-
Some info from the debug session
cat /images/4KDisk-Base-Dev/d1.original.fstypes /dev/vda3 ntfs
[Fri Jan 26 root@fogclient ~]# cat /images/4KDisk-Base-Dev/d1.partitions label: gpt label-id: 9865AAFC-B984-4860-ACF5-4D6F2513747D device: /dev/vda unit: sectors first-lba: 6 last-lba: 16777210 sector-size: 4096 /dev/vda1 : start= 256, size= 76800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=7C4743B5-7150-4672-B521-7B537528D7E7, name="EFI system partition", attrs="GUID:63" /dev/vda2 : start= 77056, size= 4096, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=F6216A84-0172-4445-B616-E36DFA20C731, name="Microsoft reserved partition", attrs="GUID:63" /dev/vda3 : start= 81152, size= 16501760, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=B6DA06DC-A5D0-434C-A6FE-494A1EFB515E, name="Basic data partition" /dev/vda4 : start= 16582912, size= 193792, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=7BB90563-4BB7-4281-98EA-3FF4BCF1FCA5, attrs="RequiredPartition GUID:63"
[Fri Jan 26 root@fogclient ~]# cat /images/4KDisk-Base-Dev/d1.minimum.partitions label: gpt label-id: 9865AAFC-B984-4860-ACF5-4D6F2513747D device: /dev/vda unit: sectors first-lba: 6 last-lba: 16777210 sector-size: 4096 /dev/vda1 : start= 256, size= 76800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=7C4743B5-7150-4672-B521-7B537528D7E7, name="EFI system partition", attrs="GUID:63" /dev/vda2 : start= 77056, size= 4096, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=F6216A84-0172-4445-B616-E36DFA20C731, name="Microsoft reserved partition", attrs="GUID:63" /dev/vda3 : start= 81152, size= 16501760, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=B6DA06DC-A5D0-434C-A6FE-494A1EFB515E, name="Basic data partition" /dev/vda4 : start= 16582912, size= 193792, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=7BB90563-4BB7-4281-98EA-3FF4BCF1FCA5, attrs="RequiredPartition GUID:63"
[Fri Jan 26 root@fogclient ~]# cat /images/4KDisk-Base-Dev/d1.fixed_size_partitions 1:2:4
[Fri Jan 26 root@fogclient ~]# cat /images/4KDisk-Base-Dev/d1. d1.fixed_size_partitions d1.minimum.partitions d1.original.swapuuids d1.shrunken.partitions d1.mbr d1.original.fstypes d1.partitions
[Fri Jan 26 root@fogclient ~]# cat /images/4KDisk-Base-Dev/d1.shrunken.partitions label: gpt label-id: 9865AAFC-B984-4860-ACF5-4D6F2513747D device: /dev/vda unit: sectors first-lba: 6 last-lba: 16777210 sector-size: 4096 /dev/vda1 : start= 256, size= 76800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=7C4743B5-7150-4672-B521-7B537528D7E7, name="EFI system partition", attrs="GUID:63" /dev/vda2 : start= 77056, size= 4096, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=F6216A84-0172-4445-B616-E36DFA20C731, name="Microsoft reserved partition", attrs="GUID:63" /dev/vda3 : start= 81152, size= 16501760, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=B6DA06DC-A5D0-434C-A6FE-494A1EFB515E, name="Basic data partition" /dev/vda4 : start= 16582912, size= 193792, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=7BB90563-4BB7-4281-98EA-3FF4BCF1FCA5, attrs="RequiredPartition GUID:63"
gdisk -l /dev/sda GPT fdisk (gdisk) version 1.0.8 Partition table scan: MBR: protective BSD: not present APM: not present GPT: present Found valid GPT with protective MBR; using GPT. Disk /dev/sda: 31246336 sectors, 119.2 GiB Model: KLUDG4UHGC-B0E1 Sector size (logical/physical): 4096/4096 bytes Disk identifier (GUID): 9865AAFC-B984-4860-ACF5-4D6F2513747D Partition table holds up to 128 entries Main partition table begins at sector 2 and ends at sector 5 First usable sector is 6, last usable sector is 31246330 Partitions will be aligned on 256-sector boundaries Total free space is 14469877 sectors (55.2 GiB) Number Start (sector) End (sector) Size Code Name 1 256 77055 300.0 MiB EF00 EFI system partition 2 77056 81151 16.0 MiB 0C01 Microsoft reserved ... 3 81152 16582911 62.9 GiB 0700 Basic data partition 4 16582912 16776703 757.0 MiB 2700
It’s a 128 GB drive, the image was a 64 GB drive, I expected it to expand to 128 GB
ntfsresize info on parts 4 and 3
ntfsresize --info /dev/sda4 ntfsresize v2022.10.3 (libntfs-3g) Device name : /dev/sda4 NTFS volume version: 3.1 Cluster size : 4096 bytes Current volume size: 793772032 bytes (794 MB) Current device size: 793772032 bytes (794 MB) Checking filesystem consistency ... 100.00 percent completed Accounting clusters ... Space in use : 14 MB (1.7%) Collecting resizing constraints ... You might resize at 13193216 bytes or 14 MB (freeing 780 MB). Please make a test run using both the -n and -s options before real resizing!
[Fri Jan 26 root@fogclient ~]# ntfsresize --info /dev/sda3 ntfsresize v2022.10.3 (libntfs-3g) Device name : /dev/sda3 NTFS volume version: 3.1 Cluster size : 4096 bytes Current volume size: 67591208960 bytes (67592 MB) Current device size: 67591208960 bytes (67592 MB) Checking filesystem consistency ... 100.00 percent completed Accounting clusters ... Space in use : 24382 MB (36.1%) Collecting resizing constraints ... You might resize at 24381546496 bytes or 24382 MB (freeing 43210 MB). Please make a test run using both the -n and -s options before real resizing!
-
In the end I am maintaining a separate image, I was able to get management to let us buy a separate surface go 4 for maintaining the 4k disk image.
I found that bhyve based VMs can be set to 4k blocks but it was cumbersome to get it to boot to fog to capture the image at the end. And when that image was deployed, it did not expand on the surface go. -
-
-
-