After updating FOG from 1.5.10.1903 to 1.5.10.2482, a subset of our UEFI clients with Realtek NICs began failing to boot, intermittently — about 6% of boots.
Clients with Intel NICs on the identical boot path are unaffected (0 failures in 62 boots). Reverting only /tftpboot/ipxe.efi to the binary shipped with 1.5.10.1903, while leaving FOG itself on 2482, stopped the failures.

The two iPXE builds involved are g5de5d (1.21.1+) and g12798 (2.0.0)`.

Environment

iPXE versions as reported by clients in the HTTP User-Agent:

• before: iPXE/1.21.1+ (g5de5d) — ipxe.efi 1,107,456 bytes
• after: iPXE/2.0.0 (g12798) — ipxe.efi 1,144,832 bytes

installfog.sh replaced every binary under /tftpboot at the time of the update.

Boot flow

DHCP (option 67) → ipxe.efi via TFTP → boot.php → exit type refind_efi → imgfetch refind.conf + chain -ar refind_x64.efi → rEFInd → local Windows Boot Manager.

rEFInd is chainloaded over HTTP (not installed on the ESP). Its config uses scanfor netboot,internal,external,optical,manual, timeout -1, use_nvram false.

Symptom

On a failing boot, every network operation succeeds:

POST /fog/service/ipxe/boot.php        200    8658 b   iPXE/2.0.0 (g12798)
GET  /fog/service/ipxe/bgdark.png      200   16523 b   iPXE/2.0.0 (g12798)
GET  /fog/service/ipxe/refind.conf     200   36624 b   iPXE/2.0.0 (g12798)
GET  /fog/service/ipxe/refind_x64.efi  200  222155 b   iPXE/2.0.0 (g12798)

…and then the client returns to PXE and repeats. Correct byte counts every time, no failed transfers, nothing in the Apache error log. Users report either a boot loop or the machine sitting on the rEFInd menu.

The failure therefore occurs after all network I/O has completed — at or after chain -ar into rEFInd, not during a transfer.

It is intermittent: the same host may succeed on one attempt and fail on the next (we observed 1 to 11 consecutive failures on one machine). Disconnecting the network cable, forcing local disk boot, always works — that is how our on-site staff were
recovering machines.

Evidence

1. Same host, same day, only the loader changed

Dell OptiPlex 3020, Realtek. From the tftpd log:

2. Old binary via per-host DHCP override

A second OptiPlex 3020 (Realtek) served the 1.5.10.1903 ipxe.efi: 2/2 unattended boots in ~18–25 s, after previously requiring a cable pull on every attempt.

3. Failure rates

Loader determined from the tftpd log — i.e. what each client actually fetched, not what the config says:

PRE-revert — /tftpboot/ipxe.efi = iPXE 2.0.0 (g12798)
  loader           NIC       ok    failed   rate
  ipxe.efi         Realtek   131      9      6.4%
  ipxe.efi         Intel      62      0      0%
  undionly.kpxe    Realtek    96      0      0%
  undionly.kpxe    Intel       5      0      0%
  snponly.efi      any         2      0      0%

POST-revert — /tftpboot/ipxe.efi = iPXE 1.21.1+ (g5de5d), FOG still on 2482
  ipxe.efi         Realtek    14      0      0%
  ipxe.efi         Intel       9      1*     (*BitLocker recovery prompt, unrelated)
  undionly.kpxe    Realtek     8      0      0%

The cleanest comparison is Intel vs Realtek on the same binary and the same UEFI path: 0/62 vs 9/140, p ≈ 0.02.

The undionly.kpxe row (0/96 Realtek) points the same way but is confounded - those clients are BIOS-mode, so they differ in boot path as well as in which driver iPXE uses. We offer it as supporting, not primary, evidence. The snponly.efi sample is too small to stand on its own.

What we ruled out

None of these discriminate between failing and working hosts:

• Machine model — OptiPlex 3020: 26 booted fine, 7 failed. Same model, both outcomes.
• BIOS version — A02 and A19 appear among failures and among successes.
• Disk type — failures include NVMe, SATA SSD and spinning HDD.
• Optical drive presence — 70% of working hosts vs 57% of failing hosts.

We also confirmed the FOG side is unchanged: $_exitTypes, $sanboot and $refind in bootmenu.class.php are byte-identical between 1903 and 2482, and refind.conf / refind_x64.efi are served byte-identically before and after (verified by response size in the access log).

Affected hardware

All report Realtek PCIe GbE Family Controller under Windows. We do not have PCI IDs to hand but can collect them.

Unaffected: Intel I219-LM (Dell Pro Slim QCS1250) — 0 failures in 62 boots.

Current workaround

We reverted only /tftpboot/ipxe.efi to the 1.5.10.1903 binary, leaving FOG on 2482. Every path we have tested works with that combination:

• normal boot and rEFInd chainload
• full image deploy — task script, 11 MB bzImage + 31 MB init.xz over HTTP, FOG client OS, NFS mount of /images, partclone, reboot into the deployed image, client check-in

Hypothesis and questions

snponly.efi — the same 2.0.0 build, but using the firmware’s UEFI SNP driver instead of iPXE’s own — works on hardware where ipxe.efi fails. That points at iPXE 2.0.0’s built-in Realtek driver. Since all network I/O completes before the failure, we suspect driver teardown/handover when chainloading another EFI application rather than anything in the transfers themselves. The intermittency suggests a race rather than a hard incompatibility.

Questions:

1. Are there known changes to the Realtek driver, or to EFI driver unbinding / shutdown() behaviour, between g5de5d and g12798?
2. Could the state iPXE leaves the NIC in affect a chainloaded EFI application - specifically rEFInd, which scans for netboot entries (scanfor netboot,...) and may re-enumerate SNP?
3. Is snponly.efi the recommended long-term choice for Realtek clients, or would you prefer a fix in the built-in driver?

We have ~1,200 Realtek clients and can reproduce readily. Happy to run test builds, capture verbose iPXE console output, or collect PCI IDs and chipset revisions.

AI-assisted-by: Claude Code (Opus 5)