@ahaeder No:
The idea of fog client is to rename the host based on what you name the computer within FOG, not the other way around.
If you want it to be a specific name, and you have the FOG Client installed, set the wanted name in the FOG UI, and the host will change automatically.
@Jeremy Thanks, the file contents are the most useful thing you’ve posted, because they show the 01-… file on your server is not FOG’s. I owe you a correction on that too: yesterday I took “the file is there” as proof the fix landed. It wasn’t — that file is yours.
1. Everything in /tftpboot/pxelinux.cfg/ is hand-written, and it’s the wrong format for pxe boot. 01-88-a2-9e-53-34-c0, boot.scr, default and default-arm all contain a U-Boot script (setenv …, tftp …, booti …). pxe boot doesn’t run scripts; it parses a PXELINUX-style config (label, kernel, initrd, append), finds no labels in yours, and does nothing. That’s your “downloads pxelinux, but then stops”, and it was never going to work regardless of the FOG bugs I fixed. Please don’t hand-write anything under pxelinux.cfg/: FOG owns the 01-<mac> names there, and its periodic reconcile deletes any 01- file whose host has no active task, so your file would vanish anyway. (boot.scr, if you ever wanted one, has to be a mkimage-wrapped binary in the TFTP root, not text under pxelinux.cfg/. You don’t need it; the pxe path is the whole thing.)
To get FOG’s file: run the updater once more (PR #1679 is merged), then in the web UI queue a Capture task for Raspberry-test. Within a moment /tftpboot/pxelinux.cfg/01-88-a2-9e-53-34-c0 will be replaced by a file starting # Generated by FOG Project with kernel arm_Image and initrd arm_init.cpio.gz, and /tftpboot/arm_Image and /tftpboot/arm_init.cpio.gz will appear beside it. Paste that file back here. Your host settings are fine: arm64, image set, primary disk /dev/sda. (acpi=off in the kernel arguments does nothing on a Pi; harmless.)
2. saveenv failed because there is no SD card, so nothing you setenv survives a reboot. That’s actually fine for the end goal: the default bootcmd=bootflow scan already does DHCP and then pxe get / pxe boot on its own, no environment needed. So for a cardless fleet you don’t need bootcmd at all. What you do need is DHCP to work inside U-Boot, which brings us to:
3. DHCP. netretry yes was doing its job (“Retry time exceeded; starting again” is it retrying, not failing). With STP confirmed on the switch, the port is most likely blocking for the first ~30 seconds after U-Boot resets the NIC. One test settles it: run setenv netretry yes then dhcp, and wait a full 60 seconds before judging. If it gets a lease on the second or third cycle, that’s STP, and the fix is PortFast / edge-port on the Pi ports (the default bootflow scan won’t retry long enough on its own, and without an SD card you can’t persist netretry). If it still has nothing after 60 seconds, then it isn’t timing: check OPNsense’s DHCP log (Services → DHCPv4 → Log) for 88:a2:9e:53:34:c0 during the attempt and tell me whether a DISCOVER arrives and whether an OFFER goes out.
Once DHCP works and FOG’s file is in place, bootflow scan (or dhcp; pxe get; pxe boot by hand) should pull arm_Image and boot FOS. If it stops anywhere after that, the last lines on screen are what I need — pasted text, not a photo, if there’s any way to capture it.
@Jeremy Good — the file being there confirms the /tftpboot fix landed. Two separate things in your report, and one of them was mine again.
“Downloads pxelinux, but then stops” — partly expected, partly another FOG bug, now fixed.
localboot 0, which tells U-Boot “nothing to do, carry on to local disk”. That’s by design: no task, no imaging. So “stops” after a successful pxe get is correct behavior unless you’d queued a capture first.http://… URLs, which is what boards with wget use. Your U-Boot’s pxe code can’t follow a URL — I checked U-Boot’s source (boot/pxe_utils.c
every kernel and initrd line is fetched over TFTP, relative to wherever the config came from, so kernel http://… became a TFTP request for a file literally called that. Fixed in working-1.6 (PR #1679): the file now says kernel arm_Image / initrd arm_init.cpio.gz, and FOG copies those two files into /tftpboot itself the first time a task is queued for an ARM host, and again whenever the kernel is updated. Run the updater once more, queue a capture for the Pi, then on the server check that /tftpboot/pxelinux.cfg/01-88-a2-9e-53-34-c0 contains kernel arm_Image and that /tftpboot/arm_Image and /tftpboot/arm_init.cpio.gz exist.The intermittent DHCP is a network-timing problem, not a FOG one, and Tom’s STP question is the right first suspect. On a switch port running classic spanning tree, the port doesn’t forward traffic for roughly 30 seconds after link-up (listening, then learning). U-Boot brings the link up and sends its DHCP discover immediately, retries for a few seconds, and gives up — that’s the “8 seconds searching”. The times it works are the times the port happened to already be forwarding. The Pi’s EEPROM boot succeeds because it retries for much longer. Two fixes, and I’d do both:
setenv autoload no
setenv netretry yes
setenv bootcmd 'dhcp; pxe get; pxe boot'
saveenv
autoload no stops dhcp from also trying to TFTP a bootfile it was never given (that’s an extra timeout and a spurious failure in your sequence). netretry yes makes U-Boot keep retrying DHCP instead of giving up after a few seconds — the right call for a headless board that has nothing else to do, but be aware it means a Pi with no DHCP server on the wire will sit there retrying rather than dropping to the prompt.
If DHCP still fails after that, the next read is OPNsense’s DHCP log for that MAC: whether the discover even arrives, and what it offers. And as before, pasted text beats a screenshot — the exact lines around pxe boot are the ones I need next.
@Jeremy Do you happen to have STP on your network?
@Jeremy Thanks for the screenshots — they changed my read of this quite a bit, and I owe you a correction first.
Correction: earlier I said pxe get “finding” pxelinux.cfg/01-88-a2-9e-53-34-c0 proved the file-generation side worked. It didn’t. U-Boot prints Retrieving file: ... before it sends the request, and the Loading: T T T T that follows means the server never answered at all. A file that really is missing gives you TFTP error: 'File not found', not timeouts. So the file-side was never proven, and it turns out it was broken on my side of the fence.
The FOG bug, now fixed: FOG was writing the 01-<mac> file under the directory it keeps the HTTP-served kernels in (/var/www/html/fog/service/ipxe/), not under /tftpboot. The TFTP daemon runs chrooted to /tftpboot, so the file existed but TFTP could never see it. That’s fixed in working-1.6 (PR #1664) with a new setting, FOG Settings → TFTP Server → FOG_TFTP_ROOT_DIR, which the installer sets to the real TFTP root. Run the updater once it’s merged and check that setting reads /tftpboot; after that, queue a task for the Pi and you should see /tftpboot/pxelinux.cfg/01-88-a2-9e-53-34-c0 appear on the server. Please confirm that file is there before the next boot test — it’s the one thing I can check in code but not on your box.
Your current boot loop is a different, earlier step. From the console shot: U-Boot itself loaded cardless, which means the Pi’s EEPROM network boot already pulled the firmware and u-boot.bin from /tftpboot over TFTP. That proves the OPNsense next-server, the FOG TFTP service and the firewall are all fine for this client — so ignore my earlier firewall angle. What’s looping is bootcmd=bootflow scan: that’s U-Boot’s standard-boot sequence, which tries mmc, usb, then ethernet, and the ethernet step starts with U-Boot’s own DHCP request (BOOTP broadcast 1, 2, 3...). That DHCP never succeeds, and it’s never been shown to work in this thread — your earlier manual test set ipaddr/serverip by hand and skipped DHCP entirely. The empty per-architecture filenames in OPNsense are fine, by the way: the pxe boot method doesn’t need a bootfile name at all. (Side note, unrelated to the Pi: the iPXE-class filename in OPNsense is what gets handed to a client that is already running iPXE, and FOG expects default.ipxe there, not ipxe.efi — with ipxe.efi your x86 UEFI clients will reload iPXE forever. Worth a look when you’re back on those.)
Two things to try at the U-Boot prompt, and please paste the exact text rather than a screenshot if you can:
dhcp
on its own. If it also loops on BOOTP, then U-Boot’s network driver isn’t getting a lease and we look at OPNsense’s DHCP log for that MAC (does the request even arrive, does it offer). If it does get an address, then:
pxe get
pxe boot
once the FOG update above is in and the 01-... file is confirmed on disk. If that works by hand, setenv bootcmd 'dhcp; pxe get; pxe boot' and saveenv gets you the automated path — bootflow scan should get there too, but the manual sequence tells us which step is at fault when it doesn’t.
@azm9s UEFI cannot be disabled or do you mean “Secure Boot” cannot be disabled? UEFI disabled might make sense, Secure Boot not being able to be disabled, does not. Otherwise vendors would not be able to load custom Firmware or fix software.
Please review your MB for secure boot and disable it. I suspect that’s the issue you’re running into.
That said, what version of FOG are you running? You said a customized FOGProject, what exactly do you mean?
If you mean:
We used FOGProject as the baseline at some snapshot in time to do our own thing with it and we have it how we like it.
Then I don’t know what you want us to do. What you have is up to you to figure out how to get working because the developers would have no idea of what your layout looks like.
@Jeremy Good — that rules out the firewall/permissions angle cleanly, thanks for actually checking rather than taking my word for it.
The new symptom is a different failure than before, and it’s outside anything FOG serves: looping on BOOTP with no attempt at Retrieving file... means it’s failing at DHCP/BOOTP negotiation, a step before pxe get would ever run. FOG can’t see that far back — the board never gets far enough to ask FOG anything.
You’ve also moved to a materially different boot path than what we’d been testing: cardless, via the Pi’s own network-boot EEPROM, rather than typing commands by hand at the U-Boot prompt. That matters here, because the EEPROM’s automatic network-boot flow runs its own boot sequence, built into that U-Boot, not necessarily the dhcp / pxe get / pxe boot lines from earlier in this thread — those were for a manually-typed bootcmd. If the EEPROM path uses bootp instead of dhcp, or expects OPNsense to hand it specific DHCP options, that’s a different thing to get right than what we tested manually.
Two things I’d need to actually say anything useful here, since I can’t see either from where I’m sitting:
nano edit touch a persisted bootcmd (printenv bootcmd), or is this the EEPROM’s own default network-boot sequence with nothing custom in the loop at all?next-server) and 67 (filename / bootfile-name) set to? The EEPROM’s PXE client needs those to know where to send its own request in the first place — if they’re pointing at the wrong place, or missing, U-Boot never gets the chance to see FOG at all, and that would produce exactly a BOOTP loop with nothing after it.That’ll tell us whether this is a boot-script problem (fixable on your end) or a DHCP-options problem (fixable in OPNsense) — right now I genuinely can’t tell which from here.
@Jeremy Correction on the firewall angle: if your server went through FOG’s own installer on ufw or firewalld, that’s actually less likely to be it than I made it sound — the installer already handles the exact “TFTP works for the request but the reply is on a random port” problem for both of those (a named tftp service on firewalld, and nf_conntrack_tftp loaded for ufw), specifically because it’s bitten people before.
So rather than “check your firewall” in general, four narrower things:
ufw, firewalld, or plain iptables? The installer only auto-configures the first two; if it’s bare iptables, my original theory stands and you’d need to allow the ephemeral UDP range by hand.lsmod | grep tftp — the installer tries to load nf_conntrack_tftp but silently continues if that fails, so it’s worth confirming it’s actually loaded rather than assumed.BOOT_external_tftp_server set to “yes” in your FOG settings? If so and your TFTP is actually running on the FOG box itself, firewalld would have skipped opening it for you.If you’ve got shell on the FOG server, tcpdump -ni <iface> port 69 or portrange 32768-60999 while you retry pxe get is still the fastest way to see whether the request goes out and nothing comes back (firewall/network) versus the request itself failing (something else entirely).
@Jeremy That’s real progress — pxe get finding pxelinux.cfg/01-88-a2-9e-53-34-c0 by MAC means the naming convention and the file-generation side both work end to end. That’s the part I couldn’t test myself, so thank you for actually running it.
On the timeout itself: “found the file, then hung mid-transfer” (Retrieving file: ... succeeding, then Loading: T T T T T T / Retry count exceeded) points at TFTP’s own connection model, not at FOG. TFTP’s initial request goes to port 69, but the server then hands the rest of that transfer off to a different, randomly chosen UDP port for the actual data — so if there’s a firewall between the Pi and your FOG server that only allows port 69 through, the request succeeds (that’s why U-Boot found the file) and every packet after it gets silently dropped. That matches your symptom closely enough that I’d check it first: either open the OS’s ephemeral UDP range to your TFTP daemon, or (simpler if this is on the same box) confirm nothing’s filtering loopback/LAN traffic to it at all. If you’ve got shell on the FOG server, tcpdump -ni <iface> port 69 or portrange 32768-60999 while you retry pxe get should show the request going out and then nothing coming back, which would confirm it.
Separately — while looking into this I found a real gap on FOG’s side and fixed it regardless of whether it’s your actual cause: the file pxe get downloads wasn’t having its permissions set after upload, so if your TFTP daemon reads as a different user than the account FOG uploads over SFTP with, that alone could produce exactly this symptom (found the file, denied reading it). That fix is in the same working-1.6 branch now, so another update will pick it up. Worth ruling both things out — they’re not mutually exclusive.
On your scalability question: for 100–200 units, don’t script the SD card at all — the RPi 4’s own boot EEPROM supports network boot as the first boot mode, ahead of SD/USB, which is exactly what you want instead of the local-storage-timeout-then-BOOTP-fallback sequence you saw. rpi-eeprom-config (or the raspi-config boot order menu) lets you set that once, and once it’s set it’s a firmware-level property of the board, not something FOG or an SD card image controls — one flash per unit, and after that every power-on goes straight to network boot, no bootcmd persistence step needed on your end at all. I’d flash that as part of whatever imaging/provisioning process gets each Pi ready before it ever talks to FOG the first time, rather than trying to solve it per-deployment.
I don’t have a Pi 4 here to confirm the exact EEPROM boot-order syntax for your firmware revision, so double-check against Raspberry Pi’s own EEPROM config docs before you commit to a fleet-wide flash — but the “make network boot the default at the firmware level” shape of the answer is what actually gets you out of typing anything at the prompt.
@Priyankha We need a lot more details.
FOG Version
What bootfile is your boot server trying to send?
Is you server sending the right FOG server for option 66/67?
What OS are you trying to capture?
What does the error show or maybe a screen shot? (UEFI Boot looks different from Legacy boot from Mac from arm, etc…)
What type of machine are you trying to image?
What have you tried?
@JJ-Fullmer @edvandro This should be fixed in dev-branch as well, but I agree with JJ here, please upgrade to working-1.6.
@Jeremy Good news on the wget front — I closed that gap I mentioned. FOG can
now serve boards with no wget at all, over plain TFTP instead of HTTP.
U-Boot’s pxe get command doesn’t need wget — every U-Boot build has it,
and unlike wget it isn’t configurable: it always fetches
pxelinux.cfg/01-<mac, dash-separated, lowercase> by TFTP from whatever
server your board’s serverip points at. FOG now keeps a real file at that
path for every host, kept in sync automatically whenever a task is queued,
completed, or canceled — so your boot sequence becomes just:
dhcp
pxe get
pxe boot
No URL, no query string, nothing for the “bad port”/no-wget class of problem
to trip over — the filename is fixed by the MAC, so there’s nothing to
substitute.
This just merged into working-1.6 — same branch you’re already tracking
(you quoted 1.6.0-beta.4707 a few hours ago; this landed after that), so
running your update again should pick it up. Two things to check once you
have:
pxe get doesn’t find the file, or finds one FOG didn’tOne thing from your last post I want to flag separately, because it looks
like a different problem from the wget one: the boot sequence you
described — times out on local storage, then falls into a BOOTP broadcast
loop that gives up with “Retry time exceeded” — reads like U-Boot’s own
default bootcmd running, not a custom one. That’s the standard
Raspberry Pi fallback behavior when nothing else answers, and it wouldn’t
run dhcp / pxe get / pxe boot at all, custom or not.
Everything you’ve described testing so far — the wget/pxe boot sequence,
the curl check, now pxe get — sounds like it’s been typed by hand at the
=> prompt each time. That proves the commands work, but it can’t be “fully
automated,” which was the goal from your very first post: nothing runs
automatically unless it’s persisted as bootcmd, either with
setenv bootcmd '...'; saveenv on the board itself, or as a boot.scr /
extlinux.conf your SD card or TFTP server hands U-Boot on its own at power-on.
Can you confirm which of those two you actually have in place right now? If
the answer is “neither yet — I’ve only been testing at the prompt,” that’s
probably the real blocker to automation, independent of wget vs. TFTP, and
worth sorting out before we look any further at the fetch mechanism itself.
@Jeremy Good, that narrows it down — no wget at all in this build, not just the older syntax. Two ways to close that:
Rebuild U-Boot with wget compiled in (keeps you on the dynamic per-host endpoint). CONFIG_CMD_WGET is a plain menuconfig option — you don’t need the HTTPS variant, which pulls in the whole lwIP/mbedTLS stack, just plain HTTP:
make rpi_4_defconfig
make menuconfig # Command line interface -> Network commands -> wget
make
Reflash the resulting u-boot.bin. Since dhcp already works for you, the network stack is there — this just turns on the one command. Once it’s in, the sequence from before should work as documented.
Or skip the dynamic endpoint and hand-roll a static boot.cmd — which is exactly what already worked for your capture: plain tftp for kernel/initrd/dtb, fixed bootargs with the task’s mac=, osid=, type=, img=, etc. baked in. The real tradeoff: tftp doesn’t ask FOG anything, so that config is identical for every board that boots it. Fine if every Pi runs the same task; not fine if you need FOG to decide per-host. boot.php exists specifically to answer “what does this MAC need right now” dynamically, and there’s currently no TFTP equivalent of it for boards that can’t do HTTP — that’s a real gap on FOG’s side, not something you’re doing wrong.
For 100+ boards I’d lean toward rebuilding U-Boot rather than going static, since you keep per-host tasking either way — but that’s your call based on how uniform the fleet’s deployment actually needs to be.
@Jeremy Two separate things here, and I think they have different answers.
On landing in debug mode: worth being precise about what isdebug actually does, because it’s not what it might look like. The code that runs your postinit script (bin/fog) doesn’t check isdebug at all — it runs whenever fog executes, whether that’s automatic or you typed fog yourself at a shell. What isdebug=yes actually does is two things: it stops FOS from launching fog automatically at boot (you land at the debug info screen and then a bare shell instead), and it turns on every debugPause() — “Press [Enter] to continue” — checkpoint throughout the entire imaging engine, not just one place. There are dozens of them. So even if you manually ran your postinit script from the debug shell and it worked cleanly, anything FOG itself does afterward (a real deploy task) is going to stop and wait for Enter repeatedly, because isdebug=yes turns all of that on at once.
That’s still coming from wherever you’ve got isdebug=yes set — either baked into your boot.cmd/append line, or as a Host Kernel Argument in the FOG web UI for this host, from when you added it to work around the multi-partition issue a few days ago. Clear it in both places. With it gone: fog runs automatically, your postinit script runs (same as before, that part was never conditional), the actual deploy runs with zero pauses, and the box reboots itself when the task completes — no interaction anywhere in the chain. That’s the automation you’re after.
On the wget “bad port” error: worth being precise about where this is happening too — it’s U-Boot itself, at the wget … | pxe boot step, before FOS is even loaded. It isn’t BusyBox and it isn’t anything in FOS or the FOG server; the config I quoted came back correctly from curl, so the server side is fine.
My best read — I don’t have your board to test this, so treat it as a hypothesis to check, not a diagnosis: U-Boot’s wget has two different implementations depending on how it was built. The newer one parses a real URL and splits host from path at the first /, so a query string with colons in it (like your MAC) shouldn’t confuse it. Older builds only understand wget <loadaddr> [<host-ip>:]<path> — no http://, no query-string awareness — and that parser looks for the first colon anywhere in the string, which lands inside mac=dc:a6:… and gets misread as a bogus port.
Can you run wget with no arguments (or help wget) at the U-Boot prompt and paste the usage line it prints? That tells us which of the two you’ve got, and whether this is fixable on your end (newer U-Boot build) or needs the TFTP-staged fallback the docs mention.
@Jeremy you seem to be trying to use fogs boot system to do your own thing. What you’re asking is perfectly possible just not via debug.
Debug has a very specific purpose so to be trying to say: “hey can you stop doing what your program is intended to do so I can do things I want using your program?” Isn’t a good use of anyone’s time, particularly the developers.
What you want to do is already possible, but you’ll have to make your own post init script.
Seeing as what you want to do is not have a fog server, but to do something else, I cannot tell you what needs to happen.
You will need to read the documentation, lax as it may be, create a custom task (not debug) to do your work autonomously, and learn how the post init scripts work.
@rogersk4132 Can you update to the latest version of working-1.6 please?
or give us information which we can use to help you more determinately?
@Jeremy Last time I told you FOG’s server had no U-Boot support and your two
options were to keep hand-rolling it or put UEFI firmware on the Pi. That’s no
longer true - I built the server side. It’s merged into working-1.6 as of
1.6.0-beta.4318, so update and you’ll have it.
There’s a new endpoint:
http://<fogserver>/fog/service/uboot/boot.php?mac=<the Pi's MAC>
It answers with an extlinux.conf-style document, computed live from the
database. If the host has a task queued you get kernel/initrd/append for FOS;
if it doesn’t, you get localboot and U-Boot falls through to the disk. Fetch it
with curl first - the whole thing is plain text and readable, which is the
point.
On the Pi:
dhcp
setenv pxefile_addr_r 0x02000000
wget ${pxefile_addr_r} http://<fogserver>/fog/service/uboot/boot.php?mac=${ethaddr}
pxe boot ${pxefile_addr_r}
Use pxe boot, not sysboot. Every extlinux example you’ll find online shows
sysboot, and it’s wrong here - sysboot reads a config off a filesystem on a
block device, so it can’t see what wget just put in memory and does nothing
visible. That’s the one I’d expect to catch people out.
This should also fix your truncation. “Unknown request type :: Null” was $type
arriving empty because your bootargs were being cut short, and I pointed at
U-Boot’s command buffer rather than the kernel. pxe boot doesn’t go through
that buffer - it parses the append line out of the loaded file straight into
bootargs, so the ceiling you were hitting isn’t in the path any more. Worth
confirming, since I’m reasoning about your build rather than looking at it.
Two deliberate omissions, both because they’re properties of your board and not
of FOG:
FOG serves no device tree. No fdt or fdtdir directive, so you keep the tree at
${fdtcontroladdr} - the one VideoCore just built for the exact board revision
it’s running on, which is better than anything I could serve you. Load
addresses stay yours for the same reason.
And FOG writes no pxelinux.cfg files. The endpoint is stateless: the board asks
by MAC and gets an answer out of the database, so there’s nothing on disk to
drift from what’s actually queued and nothing to clean up when a task finishes.
One thing to check before you start: if your FOG install has netboot set to
HTTPS, none of this will work. U-Boot’s wget is HTTP-only with no TLS at all,
so it can’t even fail a certificate check - the redirect just ends the boot with
nothing on screen. The installer now exempts service/uboot/ from the HTTP-to-
HTTPS redirect the same way it does service/ipxe/, but only when netboot is
configured for HTTP in the first place.
Full write-up is in docs/UBOOT_ARM_BOOT.md in the repo.
Being straight about what this is: the emitted config is pinned byte-for-byte
by tests and the decisions behind it are the same code the iPXE path runs, so I
know FOG emits what I intended. What I can’t tell you is that a real U-Boot
consumes it, because nobody here has a Pi. You’re the only person who can prove
that, and if it’s wrong I’d rather hear it than not.
@Jeremy Stop the dd approach - it’s what’s breaking the disk, not the Pi.
Those .img files are not raw dumps. They’re zstd-compressed partclone streams;
check any of them and you’ll see the zstd magic 28 b5 2f fd in the first four
bytes. So dd’ing d1p1.img onto /dev/sda1 writes compressed bytes straight onto
the partition, which is exactly why the firmware then can’t read it as FAT.
That’s also why partclone.restore told you it wasn’t a partclone image - it was
looking at the zstd wrapper. FOS does this at funcs.sh:847:
zstdmt -dc /images/Raspberry/d1p1.img | partclone.restore --ignore_crc -O /dev/sda1 -N -f 1
One per partition. If the image was captured with a gzip format instead, swap
zstdmt for pigz -dc.
The MBR is wrong too. d1.mbr isn’t one sector - we capture everything up to the
first partition, capped at 1MiB, so it’s usually 2048 sectors. Your count=1
threw away everything between the boot sector and the first partition. Drop the
count entirely and let it write the whole file:
dd if=/images/Raspberry/d1.mbr of=/dev/sda bs=512
Now the more useful finding. “Fatal Error: Unknown request type :: Null” is
FOS telling you $type arrived empty, and that’s the same problem as your
missing osid - your bootargs are being truncated before FOS ever reads them. It
isn’t the kernel: arm64’s COMMAND_LINE_SIZE is 2048, the same as x86, and the
bootargs line you posted is 233 characters. Look at U-Boot’s own command and
console buffer instead, that’s where the ceiling is on your setup.
Which gets to the real limitation, and I’d rather be straight about it: FOG’s
server has no U-Boot support at all. boot.php emits iPXE, and nothing generates
a boot.scr, delivers a DTB, or builds arguments for a non-iPXE client. Every
variable you’re hand-assembling is one the server would normally hand the
client, and doing it by hand is why the ordering and length problems are yours
to fight.
Two ways forward. Keep hand-rolling U-Boot, in which case use the two commands
above and keep bootargs minimal. Or put UEFI firmware on the Pi (pftf/RPi4) so
it PXE-boots into iPXE like any other machine and the server drives the whole
thing - that’s the route that already works today, and it’s where native
support would build from.
Capture works, which is the half that was broken. Deploy needs the server side
to exist.
@Jeremy That’s the first time anyone has captured an image from a Raspberry Pi
with FOG, so thank you for sticking with it through three rounds of this. The
vc4 change is merged and will be in the next build.
One correction to your boot.cmd before someone copies it: you didn’t actually
need the uInitrd wrapper. The “Wrong Ramdisk Image Format” error came from
dropping the size off the ramdisk argument - without it U-Boot reads the address
as a legacy uImage and insists on a header. Either of these works:
booti ${kernel_addr_r} ${ramdisk_addr_r}:${filesize} ${fdt_addr_r}
or the wrapper you built, which is fine as-is because you used -C none. That
part matters: -C gzip would have U-Boot decompress it and the kernel’s own gzip
support - the first bug we fixed in this thread - would never have been
exercised.
For anyone finding this later, the display problem was ours. We build without
DRM everywhere and rely on the firmware handing us a framebuffer, which is safe
on a PC because UEFI always publishes one. On a Pi it isn’t: our device tree has
no simple-framebuffer node, and the one the firmware would normally inject
disappears as soon as config.txt enables vc4-kms-v3d, which current Pi OS ships
turned on. So the kernel was booting fine the whole time and had no way to tell
anyone.
Worth saying plainly: your report exercised three separate fixes on real
hardware for the first time - the initramfs decompression, the ARM platform
support, and the NFS mount options. All of it had only ever run in QEMU before
you.
@Jeremy I found it, and it’s our bug. Our kernel is built without DRM, so it has
no vc4 driver and it depends on the firmware handing it a framebuffer the way
EFI does on a PC. On a Pi that doesn’t hold - our bcm2711-rpi-4-b.dtb has no
simple-framebuffer node, and the one the firmware would normally inject is
suppressed as soon as config.txt turns on the vc4-kms-v3d overlay, which current
Pi OS ships enabled. So our kernel had no way to write to your screen at all,
and a working boot and a dead one look exactly the same. That’s also why your
earlier test had output - kernel8.img has vc4 built in.
I’ve built the vc4 driver into our kernel. New build, same three files as
before, all from the one build:
wget https://github.com/FOGProject/fos/releases/download/EXP_20260827-114033/arm_Image
wget https://github.com/FOGProject/fos/releases/download/EXP_20260827-114033/arm_init.cpio.gz
wget https://github.com/FOGProject/fos/releases/download/EXP_20260827-114033/arm_dtbs.tar.gz
Boot it exactly the way you did last time - same load addresses, same booti
line, our DTB is fine now. One firmware note: the vc4 driver wants
avoid_warnings=2 in config.txt, otherwise the firmware can scribble over the
display setup.
All I need to know is whether you get text on the screen this time. If you do,
we’re past this and I’ll want the uname -a / cat /proc/cmdline / cat
/proc/filesystems output I asked for before. If it’s still blank, then it isn’t
the display and I’ll need serial to see what’s happening - but let’s find out
which it is first.