Unsolved Client hangs at EFI stub:
I am trying to get fog to deploy a CentOS image that I have to a server. The firmware was updated on the server and now deploy is not working. It keeps hanging at EFI stub: Measured initrd data into PCR 9
I have tried several things to try and get past this error but nothing is working. Here is my setup.
Fog version: 1.5.10
fos version: 31.03.2023 Kernel 6.1.22
ipxe version: (g0aa2e)
I believe this has to do with security but am not sure how to get past it.
Thanks for any help that I can get.
@sgilbe Set your logging level to 7 in the global FOG configuration. Lets see if that gives a bit more detail to what is going on. With logging level at 1 (default) none of the booting messages are displayed.
Just for reference what hardware are you trying to boot FOS Linux on?
@george1421 I have set the debug level to 7 under Fog Settings / Kernel Loglevel. I created a task to boot the system into debug. However it is still hanging at the same point. Where can I get the debug logs from the server? I was looking in /opt/fog/log but did not see anything that looks like a debug log. As for the type of system it is an Intel 4th Gen Xeon system (D50DNP) https://www.intel.com/content/www/us/en/products/details/servers/multi-node-server-systems/server-d50dnp.html
With the older firmware it was working and once updated to the newer firmware PXE is no longer working.
Link to Firmware release. https://www.intel.com/content/www/us/en/download/776474/bios-and-system-firmware-update-package-sfup-for-windows-and-linux-for-intel-server-d50dnp-family.html
Note to myself: Linux hangs on 4th Gen Xeon Scalable processors
I know I’ve seen posts in the forum mentioning booting issues with intel scalable processor line.
@sgilbe Can you live boot a standard linux distro like ubuntu/debian/linux mint?
@george1421 I was able to boot into the live version of ubuntu 22.04.3 desktop as well as the ubuntu 22.04.3 server installer. I was also able to boot into the Rocky Linux 9.2 installer DVD and minimum install.
@sgilbe ok so if we map this out we don’t know then if its a problem with ipxe or the FOS Linux kernel. Since other distros boot it might not be the linux kernel. The only caveat is the other distros have older kernels in the 5.x series.
This next step is to build a fos linux usb boot drive. The basis of the build is this tutorial. https://forums.fogproject.org/topic/7727/building-usb-booting-fos-image
Make sure you read the entire thread so you are aware of the caveats. When you get the boot image built, boot into the Grub menu and select the debug option. The idea is to test booting into FOS linux. If it boots then the issue is with iPXE, if it doesn’t boot then we have a linux issue.
Look at the fog forum chat for a few more hints on building this boot image.
@george1421 I will work on making the FOS usb boot drive. Will post when done.
@george1421 I have been trying to create the USB drive but am having an issue with the following line.
grub2-install --removable --no-nvram --efi-directory=/mnt --boot-directory=/mnt/boot --target=x86_64-efi
I get the error: this utility cannot be used for EFI platforms because it does not support UEFI Secure Boot.
I have checked the system that I am trying to create the USB drive on and secure boot is disabled. I have also tried to add the --force parameter but that did not work. I am using Rocky Linux 9 to create the USB drive. Maybe I am doing something wrong. I could try a different OS to do this on as well if needed.
Thanks for your help.
@george1421 Also with the system that is having issues booting into the deploy image or register host, I am able to boot into the main fog interface.
I manually Registered the host which is why it shows as registered.
I am also able to boot into the CentOS 9 Stream Installer and install that OS if I want to, as well as boot into WinPE works without any issue so I think the iPXE is working ok, but when booting the bzImage for anything but the menu it seems to hang during boot at the same screen. I have tried most of the menu items.
@sgilbe I can say I have not tested this process with Rocky Linux (any version). I did create the image for the tutorial using ubuntu/debian version. This issue may be related to a newer version of grub. You might want to try to add the --force switch to the command.
Did you look at the forum chat at the top of the page for additional hints?
@george1421 I will load ubuntu and try from that. I did try the --force switch and it still failed. I will go through the forum chat to see if there is anything there about the issue.
@george1421 Ok I have created the USB drive using Ubuntu 22.04.3 and it booted to the menu. I selected deploy / capture image and it is hung at the following spot. booting kernel… it has been sitting at that screen for 5 min.
@sgilbe ok, please pick the debug option, I think its item #6. That menu has a log level of 7.
But it does look like the linux kernel is not booting on this hardware, hopefully the additional logging level will give us some help. Also now that you have a usb drive, we can try the 5.15.x linux kernels too. It might be something in the 6.x kernel.
@george1421 After booting the USB drive I selected option 6 / debug and it is hanging at the same spot of booting kernel no extra information is displayed on the screen.
@sgilbe OK so this is telling us that the linux kernel is not booting on this hardware. Can you go to this URL https://github.com/FOGProject/fos/releases and download the 5.15.x
bzImageand copy that to the usb drive to see if 5.15 will boot on this hardware? If not working, then try the 5.10.x, if that fails then try the 4.19.x series.
As for the type of system it is an Intel 4th Gen Xeon system (D50DNP)
What exact processor is installed in this computer? I might have to search this issue based on the processor. You were able to live boot ubuntu 22.04 so a linux kernel should boot on this system. There might be something missing from the FOS linux kernel.
@george1421 Here are the CPUs that I have in the system. Intel Xeon Platinum 8480+
Here are the results from the different kernels
5.15.98 = failed
5.15.93 = failed
5.15.68 = failed
5.10.50 = failed
4.19.145 = failed
4.19.101 = failed
They all get to the same point on the screen.
OK I’ll look into the kernel configurations tonight. From what I see commercial linux can run on this hardware, there must be something missing from FOS linux. Understand that FOS Linux is a very light and fast version of linux dedicated to imaging. So it supports almost all hardware, servers them selves give us a problem since the hardware is more specific than generic.
@george1421 It looks like Ubuntu 22.04.3 installer uses kernel 6.2.0 which I was able to boot using the live usb media.
Try upgrading the PREBOOT UEFI driver which can be downloaded here: