Raspberry Pi 4 unable to pxe boot

  • Moderator

    I’ve been working on natively compiling ipxe and FOS Linux to run on the raspberry pi 4. I’ve ran into a few issues with FOG.

    First of all the raspberry pi does not support pxe booting out of the box (not a FOG issue), but the ipxe developers have a fork that supports pxe booting from a sd card https://github.com/ipxe/pipxe merging that project with FOG’s ipxe settings produces a sd card that will pxe boot into the FOG iPXE menu. Initially only did it to prove I could make it work. I just found out that the Pi 4 compute module (dedicated processor module) has an onboard MMC memory chip that can be programmed with an operating system by a convoluted process. This would be a perfect use case for FOG.

    Now to the problems.
    When you pxe boot right into the ipxe shell ${arch} is being set correctly to arm64 this is because cpuid is something only known to the ia32 and ia64 platforms. On arm64 the command fails and is unknown

    ref: default.ipxe

    cpuid --ext 29 && set arch x86_64 || set arch ${buildarch}

    So arch is being set by the ${buildarch} ipxe command. For the raspberry pi build ${buildarch} is arm64 (the version I build of iPXE)

    The first problem is that when you select hardware compatibility test from the FOG iPXE menu it attempts to load bzImage32. Inspecting the ${arch} value after the failure shows it being set to i386 by bootmenu.class.php

    Looking at bootmenu.class.php I see a reference to arm at line 146. https://github.com/FOGProject/fogproject/blob/171d63724131c396029992730660497d48410842/packages/web/lib/fog/bootmenu.class.php#L146 IMO this test should be for arm64 or arm32 or (I think) aa64 not arm.

    Edit: Looking at the linux kernel build I see the arch options are arm for 32 bit and arm64 for 64 bit.

    I think I see where i386 is getting returned to ipxe because of the error in line 517

    cpuid --ext 29 && set arch x86_64 || set arch i386


    That set arch i386 should be changed to match what default.ipxe has to offer. set arch ${buildarch}

    The same at line 542, 699, 722, 745, 796, 821, and 1905

    Fixing those lines still causes the hardware checker boot to fail with transferring bzImage32 to the target computer, but ${arch} now reports arm64 after the failure.

    Looking into this more, at line 287 https://github.com/FOGProject/fogproject/blob/171d63724131c396029992730660497d48410842/packages/web/lib/fog/bootmenu.class.php#L287 it appears to only check for x64 arch and returns bzImage32 if the x64 check fails.

            if ($_REQUEST['arch'] != 'x86_64') {
                $bzImage = $bzImage32;
                $imagefile = $init_32;

    The code could be changed to this (I know it has hardcoded values, adding fields would also require webui changes. Until official support for ARM64 is added to FOG, this will work.

            if ($_REQUEST['arch'] != 'x86_64') {
                if ($_REQUEST['arch'] = 'arm64') {
                    $bzImage = "ImagePiARM64";
                    $imagefile = "init_arm64.cpio";
                } else {
                    $bzImage = $bzImage32;
                    $imagefile = $init_32;

    Note: compiling the 5.9.8 version of linux using the configuration settings for 5.6.18 produces a linux kernel file named Image.gz and not bzImage. I’m still looking into this, but one step at a time.

  • Moderator

    @george1421 I’ve been able to boot into ipxe now, register the system with the fog server and capture an image from a raspberry pi. I have not tried to deploy an image to the pi because I’m using the sd card to boot the pi into ipxe and if it doesn’t deploy correctly I’ll have a broken boot device. I’m working on some other options right now, but basically FOG is imaging a raspberry pi running an ARM processor.

    I need to look into this a bit more, but when partclone ran the background was RED and not normally BLUE. I don’t know what this means but that is the only thing out of normal I found so far. The rest of the imaging environment seemed. Normal. The FOG iPXE menu screen displayed the background image normally and the text was colored appropriately. So the RED in partclone is the only abnormal thing I found so far.

    I did have to build a custom FOS Kernel for this Pi because the standard linux kernel could not see the USB keyboard, but using the Pi version of the linux source code did. I also had to build a ARM version of init.xz with no changes except one.

    For both bzImage and init.xz I needed to remove the compression on the files. For bzImage the bzip compression is only an x86 feature. For other architectures you need to use gzip to compress the kernel. The other thing, especially for the PIs, is that they are somewhat CPU bound so this dynamic compression takes longer to download and run than just transferring a larger uncompressed image and running it. So for both the kernel and initrd.img I’m not compressing the images. For the initrd.img I’m sending it in cpio format to the Pi.

    Development naming convention I’m using is
    kernel: ImagePiARM64
    initrd: init_arm64.cpio

    The kernel is unique to the Raspberry Pi and not intended for any other ARM architecture.

    For FOG 1.6.x we probably should add 2 new fields in the FOG Configuration -> FOG settings for the ARM kernels too just like we have for bzImage and bzImage32. But in this case ARM64 and ARM32. I can see one fog server imaging both x86 and ARM processors so we will need to make accommodations. I’m not sure how to handle the kernel field in the host configuration manager though.