@Nyeine I’ve got an Orin Nano at home and an AGX at work. I’m not sure you can image them with FOG. TBH I have never tried, so it would be interesting to find out.

One comment before I get too deep into this, if you use for your dhcp server either WIndows 2016 or later or linux as your dhcp server, you can setup profiles with your dhcp server to automatically send the right ipxe boot loader based on the pxe booting computer. You won’t need to mess with the ipxe.efi files like you have. You can surely continue to do it that way, I just wanted to let you know there might be an easier way to go about it.

So you are able to pxe boot into the iPXE boot loader, as soon as you pick a menu option that is where the wheels come off, so to say. Looking at the screen shot the process is failing at the handoff between ipxe and FOS Linux (arm_Image+arm_init). Its hard to tell if its ipxe or fos linux at fault.

I have a process to create a debug usb image for x64, but I haven’t tried to create one for arm. In theory if you have a working Jetson Orin running ubuntu you should be able to follow this to create a FOS linux usb boot image: https://forums.fogproject.org/topic/7727/building-usb-booting-fos-image In your case you don’t want to use bzImage and init.xy but use the arm_Imag… (to replace bzImage) and arm_init (to replace init.xz). The idea here is to see if you can select register from the grub menu to get FOS Linux to boot on the jetson nx. If it can then the issue is with ipxe. If it doesn’t hopefully you can debug with the usb boot image.

You may need to build a custom linux kernel for the Jetson. Just for reference https://docs.nvidia.com/jetson/archives/r36.3/DeveloperGuide/SD/Kernel/KernelCustomization.html

Same thing here with this laptop. We’ve also tested the experimental kernel but same problem.

Thanks 🙂

@yardk Okay, good news, bad news, so good news is the code loading the Arm architecture seems to be working correctly.

Bad news, we haven’t really had much time to troubleshoot against all potential arm devices and I believe this is a semi-known issue that we just haven’t been able to narrow down and/or fix at this point.

If you’re able, can you try loading a 6.1.22 arm kernel or maybe 6.1.89?

I suspect, and it’s just a sneaking suspicion, what you’re seeing is similar to the issue we’re seeing with hyper-v/proxmox and efi loading.

@ITCC Just to be clear you complied the latest version of iPXE using the instructions I provided? If that’s the case and it failed, then there is not much we can do because something in Lenovo’s firmware causing the software to hang. If rolling back the firmware isn’t an option (I don’t know what they were correcting with the update) I think we are stuck unless the iPXE developers have any idea why it might be getting stuck.

Thanks for clarifying on iPXE and FOS Linux.

@george1421 said in iPXE "Boot from hard disk" not working with RAID.:

Why refind is working is it is configured to find “windows” and boot it out of the box.

I tried configuring refind to boot grub and it partially works.
Grub shows and I can boot windows from it, but when booting Ubuntu Server I get error: failed to install protocols even though I manually installed grub with --disable-shim-lock and disabled secure boot.
Looks like it’s not secure boot related, but more like driver issue and I probably should try that iPXE chain load thing.

@george1421 said in iPXE "Boot from hard disk" not working with RAID.:

What you need is for iPXE to chain load grub.efi from the first partition so that the grub menu starts up correctly so you can dual boot.
I guess that I need to modify default.ipxe?
Could you direct me?

@rodluz , Thanks for your help, this solved the problem.

@george1421 This seems positive, looks like we may have a working solution by the end of the year. Unfortunately as the laptops we are testing this on are the CEO’s and CTO’s new ones, I will not be able to keep hold of them much longer and so they will be unavailable for testing in 2 weeks time. I will keep following the progress of this forum but wont be able to lend a hand to testing.

@Tom-Elliott The stack is as follows: There is a Debian 11 virtual system with a FOG server deployed on it. There is a DHCP server (settings in the screenshot) . There is an MSI Cube 5 12M client from which you need to make an image.
dhcp.jpg
next-server is ip address FOG server

@JJ-Fullmer Thank you for these suggestions. Yes, I am using an install package when doing it manually. I have looked all through device settings, and while I can find the punching feature, I can’t enable it.
I will look into FOG printer settings as well as FogApi powershell module. Haven’t delved into these yet.
Thanks again.

@george1421 Ok thanks for your reply. In the past i already did some research and found the arch 7 flags.

I will check out the uuid and come back with results. Thanks.

@JJ-Fullmer -y does indeed automate the schema updates.

@george1421 There seems to be an issue with the Dell boot manager.
Configuration is as follows:
Boot mode: UEFI
First boot option: Network boot
Second boot option: RAID drive (it is in HBA mode but the same behavior is observed in RAID mode as well)

Exit option from FOG GRUB is configured as EXIT right now.
Default boot option in GRUB is Boot Local Disk.

Booting the Network option is successful and GRUB menu is called. I can reimage the system as needed. But when just waiting to let it boot into the local drive it fails to recognize that it is supposed to go to a second option in the Dell boot menu, and instead of booting from SSD, it goes back to the Dell boot menu.
It can successfully boot into Windows if I choose this option directly from the Dell boot manager and if I put RAID as a first option it goes into Windows fine. But this disables FOG GRUB by default and requires manual interaction in case the system needs to be re-imaged.

I can put a PCIe NVMe drive and disable RAID completely for a sake of experiment.
On Supermicro servers FOG GRUB exits and boots into local drive just fine.

@Bristow-0 That IS very strange that going to the older kernel solved the issue. I’m glad you have it worked out, but maybe the devs need to look into the 6.x kernel a bit more. I can understand that there was a big jump in the kernel code with the 6.x release.

@Tom-Elliott
I thanks you very much,
it’s work fine now, fog can see my nvme disk
Best regards
Thierry carlotti
Corsica’s university Pascal Paoli

In the end I am maintaining a separate image, I was able to get management to let us buy a separate surface go 4 for maintaining the 4k disk image.
I found that bhyve based VMs can be set to 4k blocks but it was cumbersome to get it to boot to fog to capture the image at the end. And when that image was deployed, it did not expand on the surface go.

@jmcooper The fault probably exists with the PXE boot rom on the device.

The answer is probably a bit complicated to explain so I’ll try to keep it simple.

PXE booting can use one of the two protocols bootp or dhcp. Its up to the hardware manufacturer to decide which set of fields it looks at for the pxe boot information.

The second part is kind of a dhcp server / pxe rom issue. On each dhcp/or bootp fields there is a byte counter that tells the pxe rom how many characters are in the field name. Common convention also marks the end of a text string with a null (0x00) character. Its up to the dhcp server and pxe rom programmer to decide which method to use (I’m sure there is an RFC document that describes this). I have personally seen this in which someone was not using a mainstream dhcp server, where it only populated the byte counter field and then never terminated the string with a null character. In this case the pxe rom used the null character as end of string and the name requested from the target computer was ipxe.kpxe<with extra characters at the end until a null byte was found>

While I can explain it, it still sucks that the pxe rom firmware is doing this to you.

@flat4vw I guess a couple of things.

Instead of realtek.pxe you probably want to use realtek.kpxe.

The *?pxe files are for bios based computers. They will not boot on a uefi system.

The file is physically located in the FOG server in the /tftpboot directory.

How do you make the computer load the file? Update dhcp option 67 on your dhcp server. Either by the windows dhcp manager or if linux in /etc directory.

@ntrumpff17 The problem is that if you are using the intel hardware assisted software raid (aka intel fake raid) with the hardware controller in (Dell’s term) Raid-On mode and your computer is UEFI based, the linux kernel can’t see the drives behind this intel controller in raid mode.

Now I do have to say I have not tried with the 6.x generation of linux kernels, but the older versions could not see the drives. So the trifecta of the issue is

Intel Raid enabled + uefi firmware + linux kernel == no drive access. If you change one of the variables fog can image the computer.

The reason historically is that Intel has not released the driver interface details so that a linux driver could be built.