Unable to boot to disk after PXE Menu timeout

george1421 · Jan 20, 2022, 4:02 PM

@jmvela2x said in Unable to boot to disk after PXE Menu timeout:

Refind EFI should work though right? I’m booting PXE with legacy FW and the OS is UEFI based. This is what I see in that case:

I’m not sure I understand the statement. With a uefi system (pxe booting ipxe.efi) the exit mode should be refind. Now if you would have said bios works perfectly but uefi kind of hangs, then there are settings in the refind.conf file that we might need to tweak. But refind does do a pretty good job finding the uefi boot partition on the disk. What you have on the screen is typical of a uefi exit.

jmvela2x · Jan 20, 2022, 10:45 PM

@george1421 The OS disk is UEFI so there’s no active partition to find with SANBOOT. The NIC Oprom booting PXE is BIOS (Legacy) using undionly.kkpxe.

george1421 · Jan 20, 2022, 5:09 PM

@jmvela2x said in Unable to boot to disk after PXE Menu timeout:

The OS disk is UEFI so there’s no active partition to find with SANBOOT

I think its been a day, cause I’m not following. The disk image you provided below is surely bios. There needs to be an active partition to boot from.

Now for UEFI that is a different critter. I can’t say for sure if there is an active partition or not. The uefi firmware looks at each partition for a specific directory structure for a boot file [bootx64.efi] in an a directory called \eft\boot If the uefi firmware finds bootx64.efi it will load it which then bootstraps into the OS.

<sidebar> If you take a flash drive and format it fat32, create a directory efi\boot and then drop FOG’s ipxe.efi into that directory renamed as bootx64.efi. Now you have just created a usb boot drive into FOG imaging </sidebar>

You can image with FOG in bios mode, and lay down an image that is uefi. FOG doesn’t care about the disk structure. It only moves disks from here to there. But when you boot through PXE to the FOG menu and then onto the target OS, the boot loader you use (undionly.kpxe or ipxe.efi) needs to match the firmware mode. The boot loader (undionly.kpxe or ipxe.efi) then decides to use the exit mode SANBOOT or REFIND. You can not boot load a bios boot loader undionly.kpxe with a UEFI firmware. The same goes the other way around.

jmvela2x · Jan 20, 2022, 5:17 PM

@george1421 The disk image I provided is EFI. See EFI 100MB EFI system partition in screenshot. Since the disk structure doesn’t matter to FOG that seems to be a non-issue either way.

My BIOS has two places where I can change settings for PXE boot. In the first image you can see that boot options 1 and 4 are legacy and UEFI PXE boot respectively. UEFI is only enabled for testing and all of my troubleshooting here has been done booting to the IBA* device seen in the screenshot (what I understand to be BIOS FW for PXE booting). Booting to the UEFI PXE device fails (since the bootloader I am using is not ipxe.efi).

george1421 · Jan 20, 2022, 6:12 PM

@jmvela2x Ok… well I had a complete post where I went totally in a different direction. As part of that I grabbed disk structures from both a bios and uefi disk. As part of that I downloaded the image you provided for reference, and I’m ashamed to say that IS a uefi disk. It just looks like what ever created that disk structure had drunk monkeys in the layout creation.

Your efi partition is partition 5. While this is technically OK, it is far from standard. Below is a standard windows 10 disk format.

So all this talk about undionly.kpxe and SANBOOT can be thrown out the window. We should be working with rEFInd here. The only thing from your picture I did not see, is I guess on the system tab is the system in bios or uefi mode? I can see its pxe booting uefi. The issue with some of these advanced firmware is that you can net boot in uefi mode but have the default firmware mode as bios. You can do that with Dells that support both bios and uefi. You can take a uefi computer and via the boot menu, pxe boot in bios mode.

So now that we are dealing with a uefi computer, you should ensure in the host definition for this computer, you have the uefi exit mode as REFIND.

Just to restate what mobo this is: Gigabyte Z390 Aorus Ultra.

jmvela2x · Jan 20, 2022, 6:18 PM

@george1421 I tried that too. I posted a screenshot previously of my output for REFIND. I end up with a flashing cursor and no progress.

The system has CSM support enabled (to boot to a UEFI OS) but I am netbooting in BIOS mode using undionly.kkpxe. I will attempt to change to ipxe.efi tomorrow and try again and see what results that yields.

george1421 · Jan 20, 2022, 7:27 PM

@jmvela2x said in Unable to boot to disk after PXE Menu timeout:

I am netbooting in BIOS mode using undionly.kkpxe. I will attempt to change to ipxe.efi tomorrow

Just saying doing this will confuse things. The bios boot loader will be used (undionly.kpxe), so SANBOOT will be tried and will fail because the disk structure is efi and there is no MBR to reference. undionly.kpxe is a bios based boot loader so it can’t shell out to refind either since refind is an efi boot loader. Which is right for the disk, but can’t be started from bios (or CSM). Now the problem bits are falling in line.

So a bit off topic issue, but if your dhcp server is Windows 2012 or newer are you using profiles to send the right boot file to the target computer or are you using a static dhcp option 67? If you have a Windows 2012 or newer dhcp server there is a wiki page that shows how to setup dhcp profiles: https://wiki.fogproject.org/wiki/index.php/BIOS_and_UEFI_Co-Existence#Using_Windows_Server_2012_.28R1_and_later.29_DHCP_Policy If you go this path, make sure you activate the profile within your scope.

jmvela2x · Jan 20, 2022, 7:48 PM

@george1421 I did run across that wiki at one point. I am relatively sure our DHCP server is Windows Server 2019 at this point. Our whole organization has been moving away from 2012 for over a year now.

Since we use both types of disks, the DHCP profile sounds like the best way to go.

Just for clarification and my own curiosity, will this issue happen in reverse if I use ipxe.efi and try to sanboot to an MBR OS disk? I intend to test this either way for my own learning (as well as undionly.kkpxe with an MBR disk. Also, what is the ‘Exit’ boot option for if not to just fall back to the next boot option in BIOS?

sonic136 · Jan 21, 2022, 2:18 AM

Hi there

we’ve experienced similar issues. We updated the BIOS to the latest stable version - do version-by-version to avoid bricking the machine. Then:

disable secure boot
make sure that the boot order is set to PXE first (for obvious reasons)
set the boot mode to UEFI boot as the partition is set to EFI mode - as seen in the disk management screenshot (partition 1)
perhaps try to move your efi partition to the end of the disk - download gparted and make it a bootable flash/thumb drive and you will be able to move the efi partition. you’d want to see your disk looking something like this (attached

boot see what happens?.. ps it of course goes without saying that you should backup your image/disk before playing around with the partitions…

george1421 · Jan 21, 2022, 5:34 AM

@jmvela2x said in Unable to boot to disk after PXE Menu timeout:

Just for clarification and my own curiosity, will this issue happen in reverse if I use ipxe.efi and try to sanboot to an MBR OS disk?

The short answer here I don’t know, but I suspect it won’t work. My hesitation is that actually SANBOOT is a built in function of iPXE. Where iPXE will shell out to refind or grub exit mode functions. They are not built into iPXE but external applications. For UEFI with an exit mode of grub it will fail since grub is a bios based app. I wouldn’t think that SANBOOT called from uefi would boot into a bios formatted disk, but I simply don’t know.

I intend to test this either way for my own learning (as well as undionly.kkpxe with an MBR disk. Also, what is the ‘Exit’ boot option for if not to just fall back to the next boot option in BIOS?

I will check to see what EXIT actually does in the iPXE menu. I’m not sure what it calls, but I can find out once I get into the office.

jmvela2x · Jan 21, 2022, 9:55 PM

I moved to a Legacy image and SANBOOT exit is working with undionly.kpxe (I had previously misremembered that we were using undionlykkpxe).

I am waiting for a change request to be processed for the subnet scoping so I can test with ipxe.efi. Making progress at least.

jmvela2x · Jan 24, 2022, 5:47 PM

@jmvela2x Likewise reFIND is working with ipxe.efi and a UEFI Win10 image. Continuing testing is ongoing, but I think I may be unblocked for the moment. Thanks for the help with the sanity check @george1421.

george1421 · Jan 24, 2022, 6:30 PM

@jmvela2x Great! I’m glad you have it sorted out. FOG does work as advertised as long as you follow the rules. If you run into issues with refind, finding the EFI boot partition there are a few tweaks we can do to the refind.conf file, but mostly the default values work correctly.

jmvela2x · Jan 25, 2022, 2:30 AM

Now that I have a greater understanding I think I can pose the correct question.

Our target host need to be able to switch between BIOS and EFI based operating systems on a regular basis. I get the use case for dynamic DHCP profiles, but it seems to me that works best when each host is either EFI or BIOS based.

Let’s say for example I have an EFI version of Windows 10 on a system and reFIND is working great with ipxe.efi. After some time testing the owner of this system wants to test on Windows Server 2016, but BIOS based. Same host, different exit type.

Is there a way to make this work? I was hoping that EXIT would be the right method and would rely on falling back to the next boot device (see: exit PXE bootloader and carry on in normal boot process order).

george1421 · Jan 25, 2022, 3:23 AM

@jmvela2x said in Unable to boot to disk after PXE Menu timeout:

Let’s say for example I have an EFI version of Windows 10 on a system and reFIND is working great with ipxe.efi. After some time testing the owner of this system wants to test on Windows Server 2016, but BIOS based. Same host, different exit type.

I think you are over thinking this. How you described is how FOG works. Each host definition as well as the global definitions have 2 fields where you can set the exit types. One is for bios and one is for uefi. So lets say you have the global exit modes set as BIOS: SANBOOT and UEFI: rEFInd.

You can boot the target computer in bios mode and install a bios OS on the target computer. Every time you boot through iPXE menu, FOG will send iPXE the correct exit command [SANBOOT] (note I’m using exit as a function not the Exit method). On that same host if you switch the firmware mode to uefi and load a UEFI based OS and boot through iPXE FOG will set the exit command to [rEFInd]. You do not have to touch fog in between changing the firmware modes. You might have to touch the web ui to change the image association between bios and uefi, but you don’t need to touch the iPXE exit mode FOG will do that for you. Now if you set the exit mode in the host definition then those settings will override the global settings. That way if you have a unique host where the global settings will not work, you can change the exit modes on a host by host basis.

jmvela2x · Jan 25, 2022, 9:16 AM

@george1421 SANBOOT does nothing using the ipxe.efi bootloader and a BIOS based OS. It defaults me right back to the menu with no action taken. Also, as far as I can tell, the exit mode is based on which bootloader I am using. If I set the exit types to reFIND (EFI) and SANBOOT (BIOS) and use the ipxe.efi bootloader then it always attempts to use reFIND. This works if my OS is EFI based and boots as intended. If I am using a BIOS based OS, reFIND comes up and leaves me at its own menu. Likewise if I set the EFI exit type to SANBOOT in this same use case, the exit only serves to reset the default timer and bring me back to the main FOG menu at boot.

I was only able to successfully move to booting from the existing OS drive from the menu while using a BIOS based OS with undionly.kpxe or an EFI based OS with ipxe.efi. While that’s progress, it’s not really what I was hoping for.

george1421 · Jan 25, 2022, 5:17 PM

@jmvela2x Well this thread has been going on for 5 days now and I’m not sure I’ve done a good job explaining how FOG works.

With DHCP Profiles setup with the same computer.

BIOS Computer -> BIOS PXE Boot -> undionly.kpxe will be sent to target computer -> the global value of FOG Settings value contained in [BOOT EXIT TYPE] will be used -> SANBOOT for its Exit mode

UEFI Computer -> UEFI PXE Boot -> ipxe.efi will be sent to target computer -> the global value of FOG Settings value contained in [EFI BOOT EXIT TYPE] will be used -> REFIND_EFI for its Exit mode

This process works 99.9% of the time. The oneoffs that you might find would be UEFI based computers with quirky firmware or hardware.

Unable to boot to disk after PXE Menu timeout

145

12.0k

17.3k

155.2k