Posts made by george1421

@wmw509 said in Cannot exit IPXE menu and boot from hard drive?:

I could be wrong, but I believe iPXE is misreporting the platform.

If this is true its the first time (ever) I’ve heard of that. Could it happen, I doubt it for the simple fact that the boot loader has to be uefi or bios and it has to match the hardware or the computer doesn’t boot. A uefi based computer CAN NOT boot undionly.kpxe. Unless…

I kind of follow you on this, but am a little confused.

What I’m referring to is the firmware not the target OS. There is some firmware that will reconfigure itself on the fly based on what type of boot media is installed. I know our one Lenovo server is that way. There is no firmware mode switch inside the firmware settings, the firmware itself picks its mode.

I am using pfSense for the DHCP server and option 67 is set to unidonly.kpxe.

OK Good, with pfsense did you populate the efi fields with ipxe.efi for the 64bit and i386/ipxe.efi for the 32 bit versions of efi?

If we are still in doubt as long as the fog server, dhcp server, and pxe booting client are on the same subnet we can use tcpdump to collect what the target computer is being told to do. This is a good mystery and I’d really like to know why your system is acting the way it is. https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue grab a pcap of the pxe booting process, pxe boot to the iPXE menu is enough of a capture. Then upload the file to a file share site and post the link here and I’ll take a look at what is going down the wire. The capture filter provided will only give us the dhcp and tftp communications and nothing more.

@wmw509 said in Cannot exit IPXE menu and boot from hard drive?:

Seems like refind might not be what was causing me problems after all

As long as the platform == pcbios then is working on the bios side, but that doesn’t follow through with what you reported from inside ubuntu. Are you working with multiple hardware? OR is iPXE misreporting the platform?

OK I’m trying to think how this is possible (ipxe reporting pcbios and ubuntu reporting efi). I think I might have an idea.

There are some firmware that are dynamic depending on the boot media they will dynamically switch between bios and uefi mode depending on what they detect. If the boot media is bios based then they will switch into bios mode, if the firmware detects a efi boot media they will switch into uefi mode. There is no firmware switch its all dynamic.

So lets assume you are pxe booting this computer and you send unidonly.kpxe to this computer. This computer will see, oh that’s a bios boot loader and boot it in bios mode. FOG Imaging will work just fine in bios mode and you can deploy a uefi image with FOG Imaging in bios mode. Then after the imaging reboot, again you send undionly.kpxe to the target computer. Again it sees that is a bios boot loader so it switches to bios mode. So now you are in the iPXE menu and the menu times out to boot to the disk, but SANBOOT hangs because its trying to chain to a bios mode disk, but the disk contains a uefi boot loader. When you change the boot order so the disk is first in the order, the firmware looks to see the disk is a uefi disk and switches to uefi mode and boots from the hard drive.

I think this is how iPXE (undionly.kpxe) is reporting pcbios yet the actual OS is uefi. There is a lot of suppositions here, but it make a logical connection.

So what specifically do you have configured for dhcp option 67?

I’ve made some fundamental progress here. I remember when I was working with trying to secure boot ipxe I had 2 boot loaders the shimx64.efi and grubx64.efi that I collected from the Ubuntu repository. I also grabbed grubxnet64.efi that has network capabilities (but we don’t need that).

I copied grubx64.efi and grub.cfg from the tftpboot area over to fog/service/ipxe directory. The grub.cfg file just had a simple grub menu that allowed me to chain to ipxe.efi (not important since it was just left over code).

In the iPXE menu I created a custom menu with these parameters. The key is the second parameter of imgfetch to place the fetched file into the /grub directory in iPXE’s virtual disk.

imgfetch ${boot-url}/service/ipxe/grub1.cfg /grub/grub.cfg
chain -ar ${boot-url}/service/ipxe/grubx64.efi || goto MENU

With a uefi system I pxe booted into the iPXE menu and ran this menu. It booted grub2 and displayed my grub menu. So the framework of booting into grub2 is possible.

Using that same flow I think I can even load grub modules with something like this (untested). Once the module is loaded into the proper spot then the grub.cfg file can load them.

imgfetch ${boot-url}/service/ipxe/grub/x86_64-efi/net.mod  /grub/x86_64-efi/net.mod 
imgfetch ${boot-url}/service/ipxe/grub1.cfg /grub/grub.cfg
chain -ar ${boot-url}/service/ipxe/grubx64.efi || goto MENU

@Sebastian-Roth I grab the grub2 efi stuff with the FOS usb boot tutorial. The usb boot works for both bios and uefi. I was going to test different combinations with an ipxe menu to see if I could get it the right command line options to get it to boot https://forums.fogproject.org/topic/7727/building-usb-booting-fos-image

@wmw509 said in Cannot exit IPXE menu and boot from hard drive?:

bootx64.efi

This should be renamed to match what FOG is using the same for the 32 bit version. You can discard the bootaa version since that is for ARM processors.

I just pxe booted into ubuntu 20.04 on a uefi VM. refind found grub64.efi and booted that. So the issue might be hardware related. Describe the number of disks and their structure on this mobo? Are they sata or nvme? Do you have 2 NVMe drives installed?

@george1421 In the refind.conf (/var/www/html/fog/service/ipxe/refind.conf) There is settings
scanfor internal,external,optical,manual make sure that is set.

You might want to see if uncommenting helps uefi_deep_legacy_scan

We have also seen some disks need a little settling time so setting scan_delay 5 may help.

I may spin up a efi ubuntu system to see if I can get it to detect at leas on a VM. Ubuntu 20.04 may do things a new way.

Also surely try @JJ-Fullmer suggestion too by rolling refind back to 0.11.0 that has also helped with some mobos. I would recommend that you don’t try everything all at once

@wmw509 Sorry to keep laboring the point but knowing what mode is helpful for debugging.

If you have one of these systems booted into linux a quick check is this. ls -la /sys/firmware/efi if that file exists then its uefi boot.

And your target OS is ubuntu 20.04?

FOG has a built in function to use grub as an exit mode. Currently GRUB is only really an option for bios mode. Supporting uefi mode for grub would give us an additional exit mode where refind doesn’t work.

@wmw509 So this is a uefi based system then? If so what model computer?

Ok just for clarity there are 3 parts to pxe booting into FOG Imaging here.

1. PXE Rom in network adapter
2. iPXE Boot loader (snponly.efi in your case above). The iPXE boot loader is responsible for the iPXE menu and transferring bzImage and init.xz from the FOG server to the target computer and launching it.
3. FOS Linux (comprised of bzImage [kernel] and init.xz [virtual hard drive]). You will want to be on 5.6.18 but that doesn’t have an impact to your issue.

So you booting is failing at stage 2. The download of bzImage or init.xz is typically related to intermittent network issues or network adapter. Some of these realtek nics are finicky about the network port configuration. Check to make sure the network port you have these plugged into is configured to use one of the fast spanning tree protocols and has all of the green ethernet features disabled. There is also a realtek specific boot loader realtek.efi that you can try instead of snponly.efi.

Also make sure the bios (firmware) is up to date on this computer.

@Goll420 said in Multicasting:

Machines all fire up simultaneously and being the multicast session.

After deployment completes, they reboot and boot from network again. This takes us back to the deploy image screen as if they are all waiting for the other hosts to join the session before starting.

Right in here is when the FOG server needs to be told, “hey we are all done”, so the task on the fog server clears and when the targets reboot they just boot into the OS. That task not closing out on the fog server is why the clients are getting stuck.

@Goll420 said in Multicasting:

Yes you explained it correctly, and yes the task appears to be still running when this occurs.

Ok if this is the case and ALL target computers have completed the deployment then we will have to get the developers to look at the code that releases the task when the last system completes the deployment. This might because one of the systems didn’t wol correctly or it hung during image deployment. Because this signal is not telling the FOG server that everyone is done.

Multicast is a great bandwidth saver but it also relies on the networking infrastructure to be setup well. Also if you use multicasting ensure you have igmp snooping enabled for the VLAN where you need to multicast on.

@Goll420 Sorry I did not explain correctly. On the FOG Server in the WebUI there is a Tasks menu item. When you select that is there a task still running? If it is you can terminate it here so the computers will boot correctly. Then we need to identify why that task isn’t closing. Did all of the computers that should have been installed complete?

Will you check to see if the task is still running on the FOG server. This is done by clicking on the Tasks icon.

For these new hardware you need to use the 5.6.18 kernel found under the kernels update menu in the FOG Configuration or you can upgrade by hand using the kernel link https://fogproject.org/kernels/ I would always use an official kernel instead of any one off link you might find.

@FabianN Just to make sure we are thinking the same, you can not get to the fog iPXE menu at all right?

Another remote possibility is that you have standard spanning tree configured on the network port where you are trying to image this computer. For pxe booting FOG you should have enabled one of the fast spanning tree protocols (rstp, mstp, port-fast, etc) . Or some kind of green ethernet setting turned on. If its either of these cases you can test by putting an inexpensive (read cheap, like those home 5 port network switches) network switch between the building switch and pxe booting computer. This will mask the spanning tree issue.

There are two exit modes one for bios and one for uefi. Make sure you set the global settings to SANBOOT for bios and rEFInd for efi.

Now for bios SANBOOT works 99% of the time. For UEFI refind works (out of the box) about 90% of the time. Refind does a pretty good job of finding the EFI boot partition. You many need to tweak the refind.conf settings to have it search more locations.

So are these target system, uefi? What target OS are you trying to boot?

@John-L-Clark When I worked at a foundry in the early 1980s they use to have the saying, if something broke look for forklift tracks. Meaning, you might want to ask whoever changed something or we are going to be hunting and pecking all afternoon. Just remember that FOG doesn’t like the IP address of the FOG server changing after FOG has been installed.

Also this goes without saying you should update FOG from 1.3.5 to the latest version to support the latest hardware. 1.3.5 is pretty old, I don’t think it supports nvme disks.

@John-L-Clark FOG server on same subnet as pxe booting client?

https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue

@John-L-Clark said in FOG clients not PXE booting:

no none of our machines are booting

Ok well that changes the scope a bit.

So is the target computers on the same subnet as the FOG server?

Have your confirmed that your dhcp server has dhcp option 66 pointing to the FOG server’s IP and dhcp option 67 set to undionly.kpxe?

What device is your dhcp server? (make and model)