Error at beginning of capture--Could not fix partition layout (runFixparts)

[benc]

I’ve got a small form factor Dell PC with Windows 10 acting as the hypervisor:

• Intel Core i7-4790 @ 3.6 GHz
• 16 GB DDR3
• 256 GB Kingston SATA SSD

I’ve got a VM with Ubuntu server 24.04.1 LTS, fresh install, with all apt updates, and a fresh Fog 1.5.10.1593 install on top of that with minimal customization and configuration.

I’ve set up many Fog servers in the past, and I know that this should have been sufficient to make a working Fog system.

During my troubleshooting, I’ve set up VMs using Ubuntu 16, 18, 20, 22, and 24 to rule out any funny package version stuff. I’m seeing the exact same error/issue on all versions of Ubuntu.

After the client boots from the network and starts the script with all the partition management and prep stuff, it throws an error:

I’ve tried running a debug capture, and when I type fixparts at the prompt (after hitting enter twice) it says command not found.

I’ve tried updating the Kernel using the web GUI (http://myfogserver/fog/management/index.php?node=about&sub=kernelUpdate), but no change.

I’m currently trying the manual kernel update using the script at https://wiki.fogproject.org/wiki/index.php?title=Kernel_Update, but it’s going to take a few hours because my cellular internet connection sucks.

Maybe this is a bug where a package didn’t get included somewhere? I’ve read a little bit about fixparts not being included in some releases, but I think I also read about that being fixed. It’s not clear to me. I’m not sure how to manually slipstream a package into init.xz. I feel like that’s not what the user is expected to do here, though.

I think it’s possible that updating the kernel and inits manually at the command line might fix the problem, and I shall report back here with my results.

[benc]

I just got this error while updating init_32.xz. Now I’m wondering if my crappy internet is causing the download to time out.

root@ubuntu:/home/ben# wget https://fogproject.org/inits/init_32.xz -O /var/www/html/fog/service/ipxe/init_32.xz
--2024-09-14 21:45:37--  https://fogproject.org/inits/init_32.xz
Resolving fogproject.org (fogproject.org)... 162.213.199.177
Connecting to fogproject.org (fogproject.org)|162.213.199.177|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://github.com/FOGProject/fos/releases/latest/download/init_32.xz [following]
--2024-09-14 21:45:38--  https://github.com/FOGProject/fos/releases/latest/download/init_32.xz
Resolving github.com (github.com)... 140.82.114.4
Connecting to github.com (github.com)|140.82.114.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github.com/FOGProject/fos/releases/download/20240807/init_32.xz [following]
--2024-09-14 21:45:39--  https://github.com/FOGProject/fos/releases/download/20240807/init_32.xz
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/59623077/b104c0f7-dce0-426a-badd-6695188b4366?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240915%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240915T024540Z&X-Amz-Expires=300&X-Amz-Signature=4c6919c4de514d2eb5e9c93475af52989c33fc489abc385aef7acd0acba63aba&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=59623077&response-content-disposition=attachment%3B%20filename%3Dinit_32.xz&response-content-type=application%2Foctet-stream [following]
--2024-09-14 21:45:40--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/59623077/b104c0f7-dce0-426a-badd-6695188b4366?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240915%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240915T024540Z&X-Amz-Expires=300&X-Amz-Signature=4c6919c4de514d2eb5e9c93475af52989c33fc489abc385aef7acd0acba63aba&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=59623077&response-content-disposition=attachment%3B%20filename%3Dinit_32.xz&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.111.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 25500004 (24M) [application/octet-stream]
Saving to: ‘/var/www/html/fog/service/ipxe/init_32.xz’

/var/www/html/fog/service/ipxe/init_32.  65%[=================================================>                           ]  16.05M  8.62KB/s    in 29m 57s

2024-09-14 22:30:39 (9.15 KB/s) - Read error at byte 16826286/25500004 (Success). Retrying.

--2024-09-14 22:30:40--  (try: 2)  https://objects.githubusercontent.com/github-production-release-asset-2e65be/59623077/b104c0f7-dce0-426a-badd-6695188b4366?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240915%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240915T024540Z&X-Amz-Expires=300&X-Amz-Signature=4c6919c4de514d2eb5e9c93475af52989c33fc489abc385aef7acd0acba63aba&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=59623077&response-content-disposition=attachment%3B%20filename%3Dinit_32.xz&response-content-type=application%2Foctet-stream
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized

Username/Password Authentication Failed.

How would this be handled when the Fog installer is updating the kernel? Is there a default local set of files already in place that would be used as a fallback? If the download failed, would network booting work at all? I’m going to retry the download and see if it goes through.

[benc]

After several more dropped connections, I finally got all the files updated. Tried capturing with a Win7 client and a Win10 client, and both still give the same error. I suspect that if I set the image to RAW it might work, but I don’t have the storage space in my test environment to sufficiently test it, and a RAW image isn’t what I want anyways.

[benc]

I found a combination that worked. I installed Ubuntu Server 20.04.6 LTS, did apt updates, and installed Fog 1.5.9. Now my clients go all the way through the capture process, and everything is working as I remember.

The only thing that seems to have made a difference is using an older version of Fog, in my case 1.5.9. Hopefully this gets fixed in an upcoming release, or maybe the inits and bzimages can be fixed if that’s the problem.

[Tom Elliott]

@benc so I believe you’re having an issue and I believe you have a work around. I’m trying to think about differences and I think in your case it is specifically the kernel/init. More specifically the init as that is where the programs and scripts reside. What I am going to say is that if this were a version issue, basically all people after 1.5.10 would be broken when dealing with resizable images. Can you provide specific versions you were running and seeing these issues?

[benc]

@Tom-Elliott Sure, which versions do you need? I tried to find which build of Fog 1.5.9 I’m using in my working installation, but I’m not sure how. It wasn’t a part of the tarball filename.

I’ve got good internet now, and I’d be happy to spin up another VM to retrace my steps from earlier that resulted in a non-working install, if that would be helpful. I feel like either something is broken in the code, or I’m missing something obvious, because I’ve set up many Fog servers in the past, and I’ve never run into this issue.

[Tom Elliott]

@benc The versions of FOG you had installed, and/or specific date versions of the FOG Inits.

While we’ve been making adjustments to the code and even had an init or 2 that probably were broken for a small period of time (which we’ve since fixed) it seems you’re having the problem with the latest versions as well?

I highly doubt that it’s in the code because if it was, we’d be getting reports almost non-stop I’m sure. We have people running FOG 1.6, 1.5.10, 1.5.10.x (where x is 41 or higher on the stable sides) etc… and i’m pretty sure if resizable images were broken we’d have many more reports of this occuring.

Now that’s not to say you’re not having the problem of course, just that it feels like it was more a timing of when you got your init which may have just been missing a bit of code.

Now that I look at your picture a bit closer, though, it seems likely it’s something on this disk.

If you can upgrade to the latest stable (or 1.6 version of fog) then youu’d be willing to make this a capture debug task (just delete the current tasking, create it like you normally would, but before you just hit continue, check the box for “Schedule as debug” then hit continue.

Boot this machine, then when it gets done, it will load to a linux terminal.

From there you can try just run the fog program and press enter at each pause. it’s intended to give more details and time to look at things.

After that when this error occurs, break out with CTRL+C

Then run “fixparts /dev/sda”
Then when prompted the three commands you’ll type will be y w y

After that bring the output (either image or ssh to the machine and copy paste from a putty/ssh session - the latter being preferred for readability.

To ssh:
at the command on the host machine in debug, run passwd and create a new password.

SSH to the machine in question (us ifconfig or ip a s command to find your IP address and do the same fixparts /dev/sda command, then copy paste the output.

[benc]

Side note, it appears that the KEA DHCP server in pfSense 2.7.2 does not properly support serving the correct boot file to BIOS vs EFI systems. It all works fine with Windows DHCP. This had nothing to do with my original issue.