@mp12 Thanks for the logs. This is a bug in 1.5.10.2482, not your node passwords.

A security change in 2482 removes the storage node password from the node data that the API returns. The image and snapin replicators read their node list from that same data. So they now send an empty password, and every node rejects the login. The Undefined property: stdClass::$pass warning is that missing field.

The fix is merged to dev-branch: https://github.com/FOGProject/fogproject/pull/1770

To get it now, update from dev-branch:

cd /path/to/fogproject git checkout dev-branch git pull cd bin sudo ./installfog.sh -y

Or wait for the next stable release. Your stored passwords are correct, so you do not need to change anything on the nodes.

@AUTH-IT-Center I do believe snponly would be the recommended, rather than iPXE’s driver.

The developers at iPXE wrote the driver on their own (of course using documentation and stuff, but for all intents/purposes it is still a handrolled driver) so anything is possible.

We shipped the native iPXE 2.0.0 mainly because of the feature it allows with actual Secureboot capabilities and instead of embedding everyfile with a custom script, a more dynamic approach for when iPXE releases new version we can upgrade more easily.

For what it’s worth, I would almost want more people to default to snponly.efi (or secureboot/snponly-shimx64.efi if using/wanting secureboot after enrolling your machines of course) because this is supposed to be using the generic driver for EFI boot protocols on the NIC rather then attempting to discover the NIC using a driver loaded.

Glad you have a workaround. I think the cause is the Windows boot manager certificate change, not the image.

Your golden Optiplex installed Windows with Secure Boot on. Windows servicing then added the “Windows UEFI CA 2023” certificate to that machine’s db, and switched the boot files to a boot manager signed with it. The other Optiplex 3000s only trust the 2011 Microsoft certificates, so they reject that boot manager. bcdboot works because it copies the older 2011-signed boot manager.

Can you confirm with two checks, in admin PowerShell, on the golden machine and on one target?

[Text.Encoding]::ASCII.GetString((Get-SecureBootUEFI db).bytes) -match 'Windows UEFI CA 2023' mountvol S: /s (Get-AuthenticodeSignature S:\EFI\Microsoft\Boot\bootmgfw.efi).SignerCertificate.Issuer

If the golden machine says True and the target says False, that is the cause. A newer Dell BIOS may include the 2023 certificate in its default keys. I am also looking at having FOS add it during the Secure Boot enrollment task.

@gmaurice resetting the host encryption in the gui and then restart the fog service and it should be back up and running. You can also use the api for this, the FogApi powerhsell module (links in my signature) I have this Reset-HostEncryption function https://fogapi.readthedocs.io/en/latest/commands/Reset-HostEncryption/?h=reset+host which will also handle this reset.

Your other other option is to look into post download scripts, there’s some examples in the forums and the docs. If you’re using sysprep and unattend.xml you can inject domain join information into the unattend.xml after imaging and before windows launches for the first time, so the computer is joined to the domain before the fog service or any ui is reachable.

@Balage80 Thanks for confirming the enrollment fix.

UEFI boot: I think the cause is two new lines in default.ipxe. They read Secure Boot state from the firmware. iPXE reads it by stepping through every firmware variable, and some firmware never ends that list, so iPXE hangs there.

Please test this: take the new 979-byte default.ipxe and delete only these two lines. Keep everything else.

param secureboot ${efi/SecureBoot} param setupmode ${efi/SetupMode}

Does UEFI boot with that file? Please also post the make, model, and BIOS version of the machine. Note that re-running installfog.sh writes a new default.ipxe, which replaces a manual edit.

Pending MACs: these are not related to default.ipxe. They come from the legacy FOG Client’s Host Registration module. That module reports every adapter Windows sees, including Wi-Fi, Bluetooth, and virtual Wi-Fi Direct adapters. FOG stores each unknown MAC as pending, up to FOG_QUICKREG_MAX_PENDING_MACS per host (default 4). iPXE cannot see those adapters. You can delete the pending MACs. To stop new ones, add MAC fragments to FOG_QUICKREG_PENDING_MAC_FILTER (comma separated), or turn off Host Registration.

@kratkale Two separate things. The first gets your PCs booting today.

PXE boot, now. The installer stopped before it rewrote the boot file, so the file still says https. Change it by hand:

sed -i 's#^chain https://#chain http://#' /tftpboot/default.ipxe grep chain /tftpboot/default.ipxe

The line must now start with chain http://192.168.0.196/. Boot one PC to test it. The next complete installer run writes this file again, with http.

The installer failure. Your certificates were not changed. The installer stopped before it created anything. The Secure Boot signing files that your settings name are not on disk, so it tried to create new ones. That needs the private key of your FOG root CA, and the key is not at /etc/fog/pki/root/ca/.fogCA.key. Your root certificate is still there, so your FOG clients are not affected.

Please do not delete anything, and do not run the installer with --recreate-CA. That replaces the CA that all your FOG clients trust.

Please post the output of these commands. They show only file names and paths, no key contents:

ls -la /etc/fog/pki /etc/fog/pki/root/ca /etc/fog/pki/secureboot /etc/fog/pki/secureboot/ca /etc/fog/pki/secureboot/leaf /opt/fog/snapins/ssl/CA ls -ld /opt/fog/pki grep -E '^PKI_(root|sb)_' /opt/fog/.fogsettings find / -xdev -name '.fogCA.key' 2>/dev/null

FYI for anyone else running into this issue - I believe the problem was that I didn’t erase the /tftpboot images when I switched from http to https. I’m not really sure why that would matter, but when I did that, then re-ran the server reinstall and selected https, the images got but back in /tftpboot and my hosts didn’t see the mac error anymore.

@Priyankha We need a lot more details.

FOG Version

What bootfile is your boot server trying to send?

Is you server sending the right FOG server for option 66/67?

What OS are you trying to capture?

What does the error show or maybe a screen shot? (UEFI Boot looks different from Legacy boot from Mac from arm, etc…)

What type of machine are you trying to image?

What have you tried?

@JJ-Fullmer @edvandro This should be fixed in dev-branch as well, but I agree with JJ here, please upgrade to working-1.6.

@astrugatch Okay thanks and sorry there was that issue.

Can you pull and try installing again?

Thank you!

@Tom-Elliott

I can’t modify the company’s switches or other hardware since the system is in production. I’m currently rebuilding the FOG server on a VM on my PC and doing everything locally; it’ll be easier to troubleshoot that way. Gemini has wiped out all the previous messages and is giving me nonsense—I can’t seem to recreate the environment up to the capture stage anymore. Could you give me a rundown of everything that needs to be done—downloads, decompressing specific files in binary mode, etc.?

Thanks.

@maxcarpone I would ask you, if you’re daring/willing (it’s considered beta but uses the same pipeline with a lot more modern ui, I need people testing, and Fog_newb can likely attest the new ui look and feel though I think they ran into an issue and needed to get functional right away so the snapshotted back)

Upgrade to working-1.6.

I have a goal (along with @JJ-Fullmer ) to try to get working-1.6 to be master/stable branch by October.

1.6 has been “stagnant” since around 2017 and was in relatively stable grounds back then even.

With AI (as you undoubtly can see I’m using to help drive some things) it’s allowed us to get a lot more coding/refactoring and will hopefully present a much better experience of things on the UI side. Without testing I cannot fix UI bugs though.

AI can do some cool things, but it doesn’t know what “wrong/right” looks like, and JJ and I are only 2 people.

There are others on working-1.6 but more feedback is always good.

And I can’t add any hosts—whether via the web interface or by booting over the network—; ultimately, the new host doesn’t appear in the database…

Failed to query: Error: SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect integer value: '' for column `fog`.`moduleStatusByHost`.`msHostID` at row 1 Error Message: Error Code: "22007", Error Message: ["22007",1366,"Incorrect integer value: '' for column `fog`.`moduleStatusByHost`.`msHostID` at row 1"], Debug: SQL: [629] INSERT INTO `moduleStatusByHost` (`msHostID`,`msModuleID`,`msState`) VALUES (:hostID_0,:moduleID_0,:state_0),(:hostID_1,:moduleID_1,:state_1),(:hostID_2,:moduleID_2,:state_2),(:hostID_3,:moduleID_3,:state_3),(:hostID_4,:moduleID_4,:state_4),(:hostID_5,:moduleID_5,:state_5),(:hostID_6,:moduleID_6,:state_6),(:hostID_7,:moduleID_7,:state_7),(:hostID_8,:moduleID_8,:state_8),(:hostID_9,:moduleID_9,:state_9),(:hostID_10,:moduleID_10,:state_10),(:hostID_11,:moduleID_11,:state_11),(:hostID_12,:moduleID_12,:state_12) ON DUPLICATE KEY UPDATE `msHostID`=VALUES(`msHostID`),`msModuleID`=VALUES(`msModuleID`),`msState`=VALUES(`msState`)

@Tom-Elliott thanks Tom - using the excellent fog-docker from 88fingerslukee - so will wait to hear his response

@Tom-Elliott I did update to version 1.6.2254 and the first multicast that worked was the first one I tried after the upgrade. This second one is now hanging like the one that prompted this thread.

Where in the logs can I find any clues?

What version of FOG are you running?

@rpycroft Please attempt to update to 2231.

@toschulz So I suspect the problem is due to a divergence of db schema between dev-branch/stable + working-1.6

If you can update yoru Mysql directly:

sudo mariadb -u root fog (if no root password of course - if you have a root password add -p

UPDATE schemaVersion SET vVersion=0;

Then refresh your screen, click the install/update, and hopefully the error will go away.

The way the schema is configured is it will attempt (idempotently) to rerun every single SQL query in the schema chain.

This should fix the problem you’re seeing.

I believe there’s a column on the login history piece being requested that may just simply be missing which would cause the issue with datatables.