FOG Project Image Capture on Raspberry Pi 4 (ARM64) via U-Boot

Tom Elliott

@Jeremy The build is up, and it turned out there were two separate bugs, not one.

Bug 1: our arm64 kernel couldn’t unpack our own arm64 init. The kernel had gzip
initramfs support switched off while the init we publish is arm_init.cpio.gz. I
checked a released arm_Image with extract-ikconfig to be sure and it’s real. That
pair has been unable to boot since January 2021, when a cleanup commit pruned “unneeded
initrd compression algos” from all three kernel configs. Correct for x86, wrong for arm64,
and nobody caught it because almost nobody boots the arm64 build.

Bug 2: our arm64 kernel didn’t describe an ARM platform at all. No CONFIG_ARCH_BCM,
so no Broadcom SoC and no Raspberry Pi anything. That’s the missing NIC you found. It also
had no PL011 UART and no generic PCI host bridge, so it had no serial console and no PCI
bus on basically any arm64 machine. The config was added in 2018 by someone who said in
the commit message that they couldn’t test it, and eight years of make oldconfig carried
it forward untouched.

Both are fixed and merged. The kernel now has BCM2835/2711/2712 support, which covers Pi 3
through Pi 5: the VideoCore firmware and mailbox chain, the SD card controllers, bcmgenet,
lan78xx and smsc95xx for the older boards, macb for the Pi 5’s RP1, PCIE_BRCMSTB, and the
SoC watchdog (FOS reboots through that at the end of a task, so without it a client
finishes and just sits there).

I’ve tested it under QEMU as far as QEMU can go: the new kernel unpacks the released init
and boots all the way into FOS userspace, and with a virtio NIC it gets an IP. The old
kernel produces no console output at all on the same machine. What QEMU can’t test is
any of the Broadcom code, because its Pi 4 model disables the PCIe, genet, RNG and thermal
blocks. That’s the part I need you for. None of us has a Pi.

Testing it

Good news: don’t change your setup. Keep the U-Boot chain you already have working. We’re
not asking you to switch to iPXE yet, I just want to know whether the kernel runs on real
Pi hardware.

1. Get the files

# the new kernel
wget https://github.com/FOGProject/fos/releases/download/EXP_20260825-141730/arm_Image

# the init - unchanged, but grab this exact one, it's the pair I tested
wget https://github.com/FOGProject/fos/releases/download/EXP_20260819-141018/arm_init.cpio.gz

# device trees, if you'd rather use ours than the one you have
wget https://github.com/FOGProject/fos/releases/download/EXP_20260825-141730/arm_dtbs.tar.gz
tar xzf arm_dtbs.tar.gz          # gives you broadcom/bcm2711-rpi-4-b.dtb

Drop arm_Image and arm_init.cpio.gz into your 7efcf632/ TFTP directory. Your existing
DTB is fine, ours is just there if you want it.

2. Skip the uInitrd wrapper

You can hand the gzip file straight to booti and skip mkimage entirely:

tftp ${ramdisk_addr_r} 7efcf632/arm_init.cpio.gz
booti ${kernel_addr_r} ${ramdisk_addr_r}:${filesize} ${fdt_addr_r}

I’d rather you did it this way for this test. If you wrap it with mkimage -C gzip,
U-Boot may decompress it before the kernel sees it, and then we won’t have tested the
decompression fix at all. If you’d rather keep uInitrd, build it with -C none so
the gzip bytes get passed through untouched.

3. Move your load addresses

This one matters. Your current layout puts the DTB at 0x02600000, which is 38 MB in,
and the kernel at 0x00080000. The new arm_Image is 30 MB and its BSS runs past the end
of the file, so it can land on top of the DTB and you’d get a hang that looks exactly like
the one you’re already seeing. Give it room:

setenv kernel_addr_r  0x00080000
setenv fdt_addr_r     0x08000000
setenv ramdisk_addr_r 0x09000000

You can drop kernel_comp_addr_r and kernel_comp_size. Those are for a compressed
kernel and arm_Image isn’t one.

4. Fix the bootargs

There are four real mistakes in what you have, and one addition that will save you a lot of
time:

storagedev=/dev/sda isn’t a thing. FOS ignores it. The variable that picks a disk
is fdrive=. That’s the “Host Primary Disk” field in the web UI.
storage= is pointing at the wrong share. A capture writes to
192.168.203.113:/images/dev/. /images is exported read-only, /images/dev is the
writable one, so as written the upload will fail on permissions after a long wait.
web= needs the scheme: web=http://192.168.203.113/fog/. curl usually guesses,
don’t rely on it.
mode=debug does nothing here. FOS only reads mode= when type= is empty, so with
type=up it’s ignored. The one you want is isdebug=yes, which drops you to a shell
and pauses after every step. Use it for this first test, it’ll show you exactly where
things stop.

So:

setenv bootargs console=tty1 console=ttyS0,115200 loglevel=7 consoleblank=0 \
  isdebug=yes \
  web=http://192.168.203.113/fog/ \
  mac=dc:a6:32:c8:b4:33 osid=50 type=up \
  img=ImagePi4 imgType=mps imgPartitionType=all \
  storage=192.168.203.113:/images/dev/ storageip=192.168.203.113 \
  fdrive=/dev/sda

(console=ttyS0 is the Pi’s mini-UART. That driver is new in this build too, so if you have
a serial adapter you’ll now get output on it. Harmless if you don’t.)

5. What success looks like

Early on you should see, before anything FOG-specific:

Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 52576K
Run /init as init process

Those three lines alone tell me bug 1 is fixed on real hardware. Then the FOG banner with
Version / Init Version / Kernel Version, then Checking Operating System ... Linux, then
Attempting to check in.

If it sits on “Attempting to check in” repeating, that is a pass, not a failure. It means
everything booted and FOS is waiting for the server to hand it a job.

Then the actual capture

FOS won’t image anything until the server has a task queued for it. Hand-writing kernel
arguments doesn’t create one, which is the other reason you weren’t getting anywhere.

In the web UI, register the Pi as a host with MAC dc:a6:32:c8:b4:33.
Set that host’s Host Primary Disk to /dev/sda (that’s what fdrive= above is
doing by hand).
Create an image called ImagePi4, Linux, Multiple Partition Image - Single Disk.
On the host, Basic Tasks -> Capture.
Boot the Pi again with the same bootargs. Check-in should return and it should start
uploading.

Fair warning on one thing I can’t predict: your Pi boots off that USB SSD, and FOS is going
to be capturing the same disk it isn’t running from, which is fine. But a Pi’s boot
partition and its partition layout aren’t like a PC’s, and I don’t know yet how our
partition code handles it. That’s the second thing I want to learn from you.

What to send back

Even a failure is useful, honestly more useful than a success.

The console output from boot, as much as you can capture. Serial is easier than
photographing a screen if you have an adapter.
Specifically whether you see those three initramfs lines.
If it stops somewhere, whatever the last line on screen is. With isdebug=yes it’ll be
paused at a named step rather than just hanging, which tells me exactly where to look.

Thanks for sticking with this. Two people have asked for Pi support on here over the last
couple of years and both threads died because none of the developers has the hardware, so
you finding this and being willing to test it is genuinely how it gets fixed.

Jeremy

@Tom-Elliott Thanks for the feedback; I started the tests and got this:

Bugs 1 & 2 are 100% fixed!

The kernel successfully unpacks the initramfs (Trying to unpack rootfs image as initramfs..., Run /init as init process).
Broadcom SoC drivers, USB storage (/dev/sda - JMicron), and the internal NIC (bcmgenet -> eth0 Link is Up) are all working great.

Where it stops:
After getting the network link up, FOS fails when trying to mount with the following errors on screen:

ext3: Unknown parameter ‘nolock’
ext2: Unknown parameter ‘nolock’
ext4: Unknown parameter ‘nolock’
vfat: Unknown parameter ‘nolock’
msdos: Unknown parameter ‘nolock’
f2fs: Unknown parameter ‘nolock’

It looks like FOS is passing the NFS nolock mount option when trying to probe/mount local filesystem types or local partitions. After these errors, the Pi reboots back to the U-Boot prompt.

Let me know if you need me to test a updated init containing a fix for the mount options!

Tom Elliott

@Jeremy That mount error is our bug. FOS was passing NFS mount options without
actually telling mount it was NFS, so when the storage path doesn’t come
through, busybox falls back to trying every local filesystem in turn and each
one rejects nolock — the ext2/ext4/vfat lines were never really involved.
Fixed and merged, and there’s a new experimental build below with it in. Test it
when you get a chance and let me know how it goes.

Two things when you do. Add isdebug=yes to your bootargs — mode=debug isn’t
our debug switch, so right now you’re getting rebooted back to U-Boot before you
can read anything. And send me the output of these from the failing boot:

uname -a
cat /proc/cmdline
cat /proc/filesystems

Your error listed f2fs, and our arm64 kernel is built without f2fs and without
module support, so it can’t have produced that. I think that boot was running
your kernel8.img with our init rather than our kernel — which would mean we
still haven’t actually tested the kernel fixes on real hardware.

wget https://github.com/FOGProject/fos/releases/download/EXP_20260826-143004/arm_Image
wget https://github.com/FOGProject/fos/releases/download/EXP_20260826-143004/arm_init.cpio.gz
wget https://github.com/FOGProject/fos/releases/download/EXP_20260826-143004/arm_dtbs.tar.gz

All three are from the same build, so there is no mixing them up with what you
have now.

Jeremy

@Tom-Elliott

Hi! I downloaded the full updated pair from build EXP_20260826-143004 (arm_Image, arm_init.cpio.gz, and bcm2711-rpi-4-b.dtb from arm_dtbs.tar.gz).

I set the memory addresses exactly as requested:

kernel_addr_r: 0x00080000
fdt_addr_r: 0x08000000
ramdisk_addr_r: 0x09000000

I also passed isdebug=yes in bootargs and removed the old kernel8.img / uInitrd references to ensure only the new build files are booted.

Result on real hardware (Pi 4B):
The Pi hangs indefinitely on the 4 Raspberry logos screen with a blinking cursor, right after U-Boot executes booti. No text output appears after the logos, and I am unable to type commands or reach the FOS shell, so I cannot run uname -a or cat /proc/....

It seems the kernel halts before initializing the console/framebuffer or hangs during early memory setup on real Broadcom hardware.

Let me know if you want me to test another build or change specific earlycon/console bootargs!

Tom Elliott

@Jeremy I found it, and it’s our bug. Our kernel is built without DRM, so it has
no vc4 driver and it depends on the firmware handing it a framebuffer the way
EFI does on a PC. On a Pi that doesn’t hold - our bcm2711-rpi-4-b.dtb has no
simple-framebuffer node, and the one the firmware would normally inject is
suppressed as soon as config.txt turns on the vc4-kms-v3d overlay, which current
Pi OS ships enabled. So our kernel had no way to write to your screen at all,
and a working boot and a dead one look exactly the same. That’s also why your
earlier test had output - kernel8.img has vc4 built in.

I’ve built the vc4 driver into our kernel. New build, same three files as
before, all from the one build:

wget https://github.com/FOGProject/fos/releases/download/EXP_20260827-114033/arm_Image
wget https://github.com/FOGProject/fos/releases/download/EXP_20260827-114033/arm_init.cpio.gz
wget https://github.com/FOGProject/fos/releases/download/EXP_20260827-114033/arm_dtbs.tar.gz

Boot it exactly the way you did last time - same load addresses, same booti
line, our DTB is fine now. One firmware note: the vc4 driver wants
avoid_warnings=2 in config.txt, otherwise the firmware can scribble over the
display setup.

All I need to know is whether you get text on the screen this time. If you do,
we’re past this and I’ll want the uname -a / cat /proc/cmdline / cat
/proc/filesystems output I asked for before. If it’s still blank, then it isn’t
the display and I’ll need serial to see what’s happening - but let’s find out
which it is first.

Jeremy

This post is deleted!

Jeremy

@Jeremy said in FOG Project Image Capture on Raspberry Pi 4 (ARM64) via U-Boot:

@Tom-Elliott

GREAT NEWS! The capture completed successfully!

Here is a summary of the steps and final configuration that made it work seamlessly on the Raspberry Pi 4:

Display Fix (Kernel side):
Downloading build EXP_20260827-114033 with the integrated VC4 driver completely resolved the display freeze on the 4 raspberries logo. HDMI output initialized perfectly.

Ramdisk Format Fix (U-Boot side):
Passing raw arm_init.cpio.gz via booti was triggering a “Wrong Ramdisk Image Format” error in U-Boot. Wrapping it into uInitrd with the following command solved the issue:
mkimage -A arm64 -O linux -T ramdisk -C none -d arm_init.cpio.gz uInitrd

Final Working boot.cmd:

setenv kernel_addr_r 0x00080000
setenv fdt_addr_r 0x08000000
setenv ramdisk_addr_r 0x09000000

setenv bootargs console=tty1 loglevel=7 consoleblank=0 web=http://192.168.203.113/fog/ mac=dc:a6:32:c8:b4:33 osid=50 type=up img=ImagePi4 imgType=mps imgPartitionType=all storage=192.168.203.113:/images/dev/ storageip=192.168.203.113 fdrive=/dev/sda

tftp ${kernel_addr_r} 7efcf632/arm_Image
tftp ${ramdisk_addr_r} 7efcf632/uInitrd
tftp ${fdt_addr_r} 7efcf632/bcm2711-rpi-4-b.dtb

booti ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r}

Imaging Status:
Once booted into FOS, Partclone captured /dev/sda without any error and uploaded the full image to the FOG server!

Thank you so much for adding the VC4 driver and supporting ARM64 builds!

Tom Elliott

@Jeremy That’s the first time anyone has captured an image from a Raspberry Pi
with FOG, so thank you for sticking with it through three rounds of this. The
vc4 change is merged and will be in the next build.

One correction to your boot.cmd before someone copies it: you didn’t actually
need the uInitrd wrapper. The “Wrong Ramdisk Image Format” error came from
dropping the size off the ramdisk argument - without it U-Boot reads the address
as a legacy uImage and insists on a header. Either of these works:

booti ${kernel_addr_r} ${ramdisk_addr_r}:${filesize} ${fdt_addr_r}

or the wrapper you built, which is fine as-is because you used -C none. That
part matters: -C gzip would have U-Boot decompress it and the kernel’s own gzip
support - the first bug we fixed in this thread - would never have been
exercised.

For anyone finding this later, the display problem was ours. We build without
DRM everywhere and rely on the firmware handing us a framebuffer, which is safe
on a PC because UEFI always publishes one. On a Pi it isn’t: our device tree has
no simple-framebuffer node, and the one the firmware would normally inject
disappears as soon as config.txt enables vc4-kms-v3d, which current Pi OS ships
turned on. So the kernel was booting fine the whole time and had no way to tell
anyone.

Worth saying plainly: your report exercised three separate fixes on real
hardware for the first time - the initramfs decompression, the ARM platform
support, and the NFS mount options. All of it had only ever run in QEMU before
you.

Jeremy

@Tom-Elliott

Thank you so much for the detailed explanation and for all your hard work on this!

It’s amazing to know that this was a first for the Raspberry Pi and FOG Project, and I’m really glad my tests helped validate those three major fixes on real hardware.

Good catch on the :filesize argument syntax for booti as well. That clarification will definitely be super helpful for anyone setting up ARM64/Pi deployments in the future!

Everything is working smoothly on my end now. Thanks again for the incredible reactivity and support.

The next test is to deploy the image to other Raspberry Pis; I’ll keep you posted.

Jeremy

@Tom-Elliott

I’ve been working on deploying images to a Raspberry Pi 4 (ARM64) using FOG (version 1.5.10.2254, FOS kernel 6.18.38), and I wanted to share our findings, encountered errors, and current limitations we faced during the process. Hopefully, this can help improve native ARM support or guide others trying to achieve unattended RPi deployments.

Here is a summary of what we experienced and how we bypassed it:

U-Boot bootargs Buffer Overflow

Issue: When adding extra environment variables to bootargs inside boot.cmd (such as custom storage IPs, disk targets, or MAC addresses), the ARM64 kernel command line exceeded the maximum allowed size. This caused the kernel boot process to silently halt right after the network interface came up (Link is Up).

Workaround: Kept the bootargs string as minimal as possible and relied on FOG web settings where applicable.
Missing osid Parameter

Issue: FOS initialization stops with No os id passed (determineOS) and reboots if the kernel parameters do not explicitly provide osid=50 (or the corresponding OS ID) alongside type=down.

Workaround: Explicitly appending osid=50 type=down in the U-Boot script parameters.
FOG Automatic Script Orchestration vs. ARM Multi-Partition Layout

Issue: Even when successfully booting into FOS and passing the correct variables (img=Raspberry, storage=…, fdrive=/dev/sda), the automated fog script skips or fails to correctly parse multi-partition layouts on USB/SD external drives (/dev/sda), directly jumping to “Task Complete” without actually writing data via Partclone or triggering errors like Fatal Error: Unknown request type :: Null.

Workaround: We had to enter FOG Debug Mode (isdebug=yes), export variables manually (export storage, export img, export type=down), and notice that FOG’s single-partition versus multi-partition routines expect specific raw file structures (d1.mbr, d1p1.img, d1p2.img).
Raw Image Files (.img) vs. Partclone Format

Issue: When attempting manual deployment via partclone.restore, Partclone throws This is not partclone image because FOG stores standard multi-partition RPi captures as raw binary dumps (.img) rather than compressed partclone streams.

Workaround: Restoring the MBR and partitions required direct block-level commands:
Bash

dd if=/images/Raspberry/d1.mbr of=/dev/sda bs=512 count=1
dd if=/images/Raspberry/d1p1.img of=/dev/sda1 bs=4M status=progress
dd if=/images/Raspberry/d1p2.img of=/dev/sda2 bs=4M status=progress

Result: While the partitions are written, the RPi firmware still struggles to directly boot the resulting USB structure (Unable to read partition as FAT), indicating that standard FOG MBR generation (d1.mbr) doesn’t align cleanly with RPi’s expected GPT/FAT bootloader requirements for USB-MSD booting.

It seems the automated deployment engine (fog script) inside FOS needs specific adaptations for ARM/Raspberry Pi multi-partition disk imaging (handling fdrive=/dev/sda and raw dd-like partition layouts seamlessly).

Any insights or recommendations on how to properly handle native RPi deployments through FOG would be greatly appreciated!

Tom Elliott

@Jeremy Stop the dd approach - it’s what’s breaking the disk, not the Pi.

Those .img files are not raw dumps. They’re zstd-compressed partclone streams;
check any of them and you’ll see the zstd magic 28 b5 2f fd in the first four
bytes. So dd’ing d1p1.img onto /dev/sda1 writes compressed bytes straight onto
the partition, which is exactly why the firmware then can’t read it as FAT.
That’s also why partclone.restore told you it wasn’t a partclone image - it was
looking at the zstd wrapper. FOS does this at funcs.sh:847:

zstdmt -dc /images/Raspberry/d1p1.img | partclone.restore --ignore_crc -O /dev/sda1 -N -f 1

One per partition. If the image was captured with a gzip format instead, swap
zstdmt for pigz -dc.

The MBR is wrong too. d1.mbr isn’t one sector - we capture everything up to the
first partition, capped at 1MiB, so it’s usually 2048 sectors. Your count=1
threw away everything between the boot sector and the first partition. Drop the
count entirely and let it write the whole file:

dd if=/images/Raspberry/d1.mbr of=/dev/sda bs=512

Now the more useful finding. “Fatal Error: Unknown request type :: Null” is
FOS telling you $type arrived empty, and that’s the same problem as your
missing osid - your bootargs are being truncated before FOS ever reads them. It
isn’t the kernel: arm64’s COMMAND_LINE_SIZE is 2048, the same as x86, and the
bootargs line you posted is 233 characters. Look at U-Boot’s own command and
console buffer instead, that’s where the ceiling is on your setup.

Which gets to the real limitation, and I’d rather be straight about it: FOG’s
server has no U-Boot support at all. boot.php emits iPXE, and nothing generates
a boot.scr, delivers a DTB, or builds arguments for a non-iPXE client. Every
variable you’re hand-assembling is one the server would normally hand the
client, and doing it by hand is why the ordering and length problems are yours
to fight.

Two ways forward. Keep hand-rolling U-Boot, in which case use the two commands
above and keep bootargs minimal. Or put UEFI firmware on the Pi (pftf/RPi4) so
it PXE-boots into iPXE like any other machine and the server drives the whole
thing - that’s the route that already works today, and it’s where native
support would build from.

Capture works, which is the half that was broken. Deploy needs the server side
to exist.

FOG Project Image Capture on Raspberry Pi 4 (ARM64) via U-Boot

Testing it

1. Get the files

2. Skip the uInitrd wrapper

3. Move your load addresses

4. Fix the bootargs

5. What success looks like

Then the actual capture

What to send back

89

12.7k

17.6k

157.0k