@Tom-Elliott

I can’t modify the company’s switches or other hardware since the system is in production. I’m currently rebuilding the FOG server on a VM on my PC and doing everything locally; it’ll be easier to troubleshoot that way. Gemini has wiped out all the previous messages and is giving me nonsense—I can’t seem to recreate the environment up to the capture stage anymore. Could you give me a rundown of everything that needs to be done—downloads, decompressing specific files in binary mode, etc.?

Thanks.

@kratkale 09-08-26 seems to me that the FOGMulticastManager service isn’t started or died somewhere.

Can you run:

sudo systemctl restart FOGMulticastManager sleep 5 sudo systemctl -l status FOGMulticastManager

On a separate window it might be helpful to see your php-fpm www-error logs (see my footer to see where to find that information)

@maxcarpone I would ask you, if you’re daring/willing (it’s considered beta but uses the same pipeline with a lot more modern ui, I need people testing, and Fog_newb can likely attest the new ui look and feel though I think they ran into an issue and needed to get functional right away so the snapshotted back)

Upgrade to working-1.6.

I have a goal (along with @JJ-Fullmer ) to try to get working-1.6 to be master/stable branch by October.

1.6 has been “stagnant” since around 2017 and was in relatively stable grounds back then even.

With AI (as you undoubtly can see I’m using to help drive some things) it’s allowed us to get a lot more coding/refactoring and will hopefully present a much better experience of things on the UI side. Without testing I cannot fix UI bugs though.

AI can do some cool things, but it doesn’t know what “wrong/right” looks like, and JJ and I are only 2 people.

There are others on working-1.6 but more feedback is always good.

@Tom-Elliott Thank you, tested, confirmed it’s fixed.

Now I read the AI summary and got what I need:

It stores one row for each address in the hostNetwork table. Limits of this method The sleeping host must run fog-agent. The server uses the sleeping host’s own last report to find its subnet. A host with the legacy FOG Client, or with no client, has no rows, so the relay cannot help it. The old path still runs for it.

So If I populate the hostNetwork table for the hosts that I need manually, It will work. Thank you for the details.

@ahaeder A final setting - you have to set Bios exit type = GRUB (not the default SANBOOT) or the boot hangs at ‘Booting from SAN device’.

@azm9s Good that legacy boot works now. For UEFI, your first tcpdump already shows one problem. It stops UEFI clients on every board, with Secure Boot on or off.

First: remove option 60 from your Windows DHCP server.
Your DHCP offer contains option 60 = “PXEClient”. UEFI firmware reads this as “this server is a PXE boot server”. It sends a second request to 192.168.65.35 on UDP port 4011 and waits. FOG does not listen on port 4011, so the firmware stops before it downloads any file. That is why no boot file worked. Legacy network boot does not do this, so legacy works. You only need option 60 if WDS runs on the same server as DHCP.

Second: set option 67 for UEFI to secureboot/snponly-shimx64.efi.
Your version (1.5.10.2253) installs this file. Check that it is there:

ls /tftpboot/secureboot/

It is signed by Microsoft, so it boots with Secure Boot on or off. You do not need to disable Secure Boot. Do not point option 67 at secureboot/ipxe.efi directly. With Secure Boot on, the firmware rejects it. If the menu loads but the network does not start, use secureboot/ipxe-shimx64.efi instead.

To keep legacy and UEFI working at the same time, use Windows DHCP policies. Vendor class PXEClient:Arch:00000 keeps your current legacy file, and PXEClient:Arch:00007 gets the file above:
https://docs.fogproject.org/en/latest/kb/how-tos/bios-and-uefi-co-existence

If UEFI still fails after these two changes, run this on the FOG server while the PC boots, and post the output:

tcpdump -i eth0 -n -vv ether host fc:9d:05:76:7c:00

@astrugatch Okay thanks and sorry there was that issue.

Can you pull and try installing again?

Thank you!

@jmeyer I have some e16 g1s and I am not having the same issue. What version of fog are you running? Are there any other secure boot settings you have configured or maybe not configured? Can you get to the mok enroll with secure boot off? And then turn it back on?

@servicedesk-pianezza Those are clean tests and they kill my theory. Correcting the record, and then I think your
own last test moves this a long way.

The stale-metadata idea is dead. A zeroed disk with a fresh deploy still hangs, and
mdadm --examine found no superblock either side of the wipe. Drop it.

I also replayed our GPT restore path locally against a Windows 11 resizable image, onto a
disk deliberately smaller than the captured one — the same order FOS uses: dd of d1.mbr,
sgdisk -z, sgdisk -gl, then the filldisk table through sfdisk. The result verifies clean:
sgdisk -v reports no problems, the protective MBR is a single 0xEE entry spanning the whole
device, and first/last usable sectors match the target. So a malformed partition table is not
what we are looking at either.

Now the part I think you undersold. You reached the FOG iPXE menu and chose “Boot from hard
disk”, with the deployed drive fitted, and then it hung. That means the firmware finished
POST, brought up the NIC, ran iPXE and drew a menu — all with that drive present. The
firmware is not the thing that hangs. It hands off to bootmgfw.efi and the hang is after
that point.

Which reframes the symptom. “Stuck at the Dell logo” is not the firmware stalling. It is the
Windows boot chain hanging before anything repaints the screen, so the OEM logo simply stays
up. F12 being dead is expected there — the firmware gave up the keyboard at handoff. It also
explains the missing Automatic Repair: Windows’ boot-failure counter is incremented by the
boot manager, and a hang never reaches the code that does it.

So the question is now why this image hangs in early Windows boot on a Whiskey Lake Latitude
and not on a 12th-gen HP. Two things to do, in this order.

Deploy the same image to the same Dell as Single Disk (Not Resizable). This splits the
problem in half for the cost of one deploy. Both 2020 reports on this hardware said
non-resizable worked where resizable did not — see banana123 in topic 14147, “Using Multiple
Partition Image - Single Disk (Not Resizable) DOES work fine”. If non-resizable boots, the
fault is in our resize path, it is ours, and I will want d1.minimum.partitions,
d1.fixed_size_partitions and a full debug-deploy transcript. If it hangs too, the resize
path is exonerated and it is the image.

Make Windows tell you where it stops. Boot a Windows installer USB on the hung machine,
Shift-F10 for a command prompt, find the ESP letter with diskpart, then:

bcdedit /store X:\EFI\Microsoft\Boot\BCD /set {default} sos on bcdedit /store X:\EFI\Microsoft\Boot\BCD /set {globalsettings} bootmenupolicy legacy

sos replaces the logo with the list of boot drivers as they load, so the screen names the
last thing it got to instead of showing you a logo. bootmenupolicy legacy gives you the F8
menu, and Safe Mode is itself a useful result.

Last question, because you have not said it anywhere in the thread: was the image captured
after sysprep /generalize /oobe /shutdown, or from a VM that had simply been shut down? An
image captured without generalize carries the source machine’s driver and device state, and
booting on one chipset but not another is the usual way that shows up.

@gmaurice resetting the host encryption in the gui and then restart the fog service and it should be back up and running. You can also use the api for this, the FogApi powerhsell module (links in my signature) I have this Reset-HostEncryption function https://fogapi.readthedocs.io/en/latest/commands/Reset-HostEncryption/?h=reset+host which will also handle this reset.

Your other other option is to look into post download scripts, there’s some examples in the forums and the docs. If you’re using sysprep and unattend.xml you can inject domain join information into the unattend.xml after imaging and before windows launches for the first time, so the computer is joined to the domain before the fog service or any ui is reachable.

@Balage80 Thanks for confirming the enrollment fix.

UEFI boot: I think the cause is two new lines in default.ipxe. They read Secure Boot state from the firmware. iPXE reads it by stepping through every firmware variable, and some firmware never ends that list, so iPXE hangs there.

Please test this: take the new 979-byte default.ipxe and delete only these two lines. Keep everything else.

param secureboot ${efi/SecureBoot} param setupmode ${efi/SetupMode}

Does UEFI boot with that file? Please also post the make, model, and BIOS version of the machine. Note that re-running installfog.sh writes a new default.ipxe, which replaces a manual edit.

Pending MACs: these are not related to default.ipxe. They come from the legacy FOG Client’s Host Registration module. That module reports every adapter Windows sees, including Wi-Fi, Bluetooth, and virtual Wi-Fi Direct adapters. FOG stores each unknown MAC as pending, up to FOG_QUICKREG_MAX_PENDING_MACS per host (default 4). iPXE cannot see those adapters. You can delete the pending MACs. To stop new ones, add MAC fragments to FOG_QUICKREG_PENDING_MAC_FILTER (comma separated), or turn off Host Registration.

OK that makes sense. This got me going for driver deployment. Thank you for your help!

@Tom-Elliott I edited the sshd_config file and changed
/usr/lib/openssh/sftp-server
to
internal-sftp

I’m not sure how the installer failed to make this accomodation in the first place but I’m glad it appears to be working. I deployed the image to another laptop to make sure everything is working. It looks like the problem can be considered resolved now. Thanks for your help!

@jmeyer Would you be willing/able to try again without bypassing it to help confirm the fix?
@rogersk4132 thank you for testing and confirming!

Awesome and thank you. Confirmed working.

Hi Tom,

In my case the culprit was /images/postdownloadscripts/fog.postdownload.

echo "Activating boot partition..." sfdisk --activate /dev/sda 1 echo "Rebooting..." reboot -f

Fix was simply commenting out (or deleting) those lines. After cleanup the file looks like this:

cat /images/postdownloadscripts/fog.postdownload #!/bin/bash ## This file serves as a starting point to call your custom postimaging scripts. ## <SCRIPTNAME> should be changed to the script you're planning to use. ## Syntax of post download scripts are #. ${postdownpath}<SCRIPTNAME>

Thanks for the pointer, Tom — saved me from chasing GPT partition tables for a problem that had nothing to do with them.

Glad you have a workaround. I think the cause is the Windows boot manager certificate change, not the image.

Your golden Optiplex installed Windows with Secure Boot on. Windows servicing then added the “Windows UEFI CA 2023” certificate to that machine’s db, and switched the boot files to a boot manager signed with it. The other Optiplex 3000s only trust the 2011 Microsoft certificates, so they reject that boot manager. bcdboot works because it copies the older 2011-signed boot manager.

Can you confirm with two checks, in admin PowerShell, on the golden machine and on one target?

[Text.Encoding]::ASCII.GetString((Get-SecureBootUEFI db).bytes) -match 'Windows UEFI CA 2023' mountvol S: /s (Get-AuthenticodeSignature S:\EFI\Microsoft\Boot\bootmgfw.efi).SignerCertificate.Issuer

If the golden machine says True and the target says False, that is the cause. A newer Dell BIOS may include the 2023 certificate in its default keys. I am also looking at having FOS add it during the Secure Boot enrollment task.

@Tom-Elliott That was quick 😊 I just tested and it works as expected. Thank you.

Rahman

FYI for anyone else running into this issue - I believe the problem was that I didn’t erase the /tftpboot images when I switched from http to https. I’m not really sure why that would matter, but when I did that, then re-ran the server reinstall and selected https, the images got but back in /tftpboot and my hosts didn’t see the mac error anymore.