Hi @Tom-Elliott
After moving from RC-2 to RC-4 the plugin, the WinPE image and the step that closes the task at the end of the WinPE flow all kept working. Only the Apache part needed a change.

With RC-4 the whole vhost sits inside FOG’s managed block, so my earlier Alias plus rewrite exception was gone and /fog/winpe_3400/wimboot started answering 308 (FOG’s rewrite rule sends it to the API). Instead of putting the exception back, I now serve the files from /winpe_3400/ (no /fog/ prefix), with an Alias and a <Directory> in a separate conf-available file enabled with a2enconf. FOG’s rewrite rule only matches /fog/, so it never sees these requests, and nothing in 001-fog.conf should need restoring after an update. The only change in the plugin is its WIMBOOT_HTTP_PATH constant.

Checked after the change: the Latitude fetches wimboot, BCD, boot.sdi and boot.wim from the new path (all 200), WinPE starts, and a full deploy on the Latitude 3400 and one on an HP with the normal FOS/Partclone flow both completed and closed their tasks.

Hi @Tom-Elliott,
follow-up on this thread: I never found the root cause of the intermittent firmware hang on the Latitude 3400 (16 hypotheses ruled out, including the Protective MBR fields we discussed, thanks again for the help with that), but deploying these machines with diskpart + DISM from a WinPE booted via wimboot (through a small FOG plugin) has worked reliably so far, while every other model stays on FOS/Partclone. The attached report covers the investigation, the workaround and a few side findings (WinPE deploys leaving the task queued, Secure Boot/MOK, a vsftpd home-directory pitfall). If anyone has seen the same hang or has ideas about the cause, I’d like to hear it.Report_Latitude3400_FOG_BootIssue_EN.md.txt

Thanks for your help

Hi @Tom-Elliott
Finally have a clean, confirmed answer — and I apologize for the back-and-forth on disk labeling in my last few posts, that was entirely on me getting physical drives and PowerShell disk numbers crossed. This is now verified cleanly across two different physical NVMe drives (SSSTC and Toshiba), each tested in both states:

FOG-deployed Windows 11 (either drive) → Protective MBR (LBA0) has:

• Real x86 bootstrap code present (the classic “Invalid partition table” / “Missing operating system” stub)
• Partition entry: End CHS = FF FF FF (sentinel), Size field = exact sector count (e.g. AF 32 CF 1D)
  → Hangs at F2/Setup and normal boot, on both drives, reproducibly.

Same drive, wiped and repartitioned natively via Windows diskpart (clean, convert gpt, create partition efi/msr/primary), then had real Windows boot files copied onto the ESP via robocopy (so it’s not just an empty table — it has genuine bootmgfw.efi, CIPolicies, language resources, etc.) → Protective MBR has:

• Boot code region entirely zeroed
• Partition entry: End CHS = FE 7F 99 (specific computed value), Size field = FF FF FF FF (sentinel)
  → Boots/reaches F2 Setup reliably, every time, tested repeatedly on both drives.

So the variable isn’t the drive brand, isn’t the GPT header (identical format on both), isn’t the partition table entries (identical GUIDs/types/order) — it’s specifically how the Protective MBR at LBA0 is written. sgdisk/gdisk (what FOG’s restore uses) writes a real bootstrap stub (likely copied forward from the source Windows install during Partclone’s capture, since GPT disks normally don’t need real MBR boot code) combined with an exact size field. diskpart writes it the opposite way: zeroed boot code, sentinel size, specific CHS.

Given the 3400’s firmware hangs specifically at “Preparing to enter BIOS Setup” (confirmed earlier — this is firmware disk enumeration, before any OS involvement) when it encounters the sgdisk-style Protective MBR, my best guess is a legacy/CSM-compatibility path in this BIOS reads that boot code region and/or the exact-size field and gets stuck — possibly trying to validate or execute the bootstrap stub even though the system is UEFI-only, or choking on an exact size value in a field it expects to be a sentinel.

Question for you: does FOG’s restore process (or the underlying sgdisk/gdisk call) preserve/write real x86 boot code into the Protective MBR, and is there a flag to zero that region and/or force the sentinel size convention instead? If sgdisk has an option for this (or if it’s something Partclone carries over from the source capture rather than something sgdisk actively writes), that would be great to know — happy to test a patched version if you can point me to where in the FOG scripts this happens.

This feels like a genuinely actionable, reproducible finding now — thanks for sticking with the bisection methodology, it’s what got us here.

Thanks

Hi @Tom-Elliott
Update on the bisection: re-deploying and re-testing multiple times shows this is NOT deterministic even with what should be an identical table each time. Most attempts hang at F2, but occasionally one boots through — same image, same deploy process, same resulting table.

So sgdisk -Z (empty table) → 100% reliable, boots/F2 every time.
Populated table (Win11 deploy) → hangs most of the time, but not always — same exact deploy repeated shows different outcomes.

Given that, I don’t think this is “a wrong byte value” in the table (that would be reproducible 100% of the time either way). This looks more like the Latitude 3400 firmware having a genuine timing/race issue enumerating a populated multi-partition GPT NVMe at POST — mostly failing, occasionally succeeding, regardless of the specific bytes. That would also explain why the two 2020 forum reports on this same hardware family “fixed” it by wiping on another machine: not because specific stale bytes were the cause, but because an empty/simple table happens to avoid triggering whatever race condition the firmware has with a fully populated one.

If that’s right, this isn’t something byte-level bisection will resolve — it’s a firmware reliability issue with this NVMe controller + BIOS 1.39.0 combination when handling multi-partition GPT disks, independent of FOG, the OS, or the capture pipeline. I don’t have a way to test that hypothesis further without either a Dell hardware diagnostic tool or a firmware engineer’s tools.

At this point, given BIOS 1.39.0 is the latest available for this model, I don’t think there’s a software fix on my end. Documenting this for anyone else who hits it on Latitude 3400 with an NVMe-heavy image: expect intermittent F2/boot hangs with populated GPT tables on this platform, and budget for multiple retry attempts as a practical workaround, since a full wipe-and-redeploy doesn’t reliably avoid it either (confirmed just now — it just probabilistically improves odds by luck, not a guaranteed fix).

Thanks for pushing the bisection methodology — even though it landed on “flaky firmware” rather than a fixable root cause, it at least closes the loop definitively and rules out FOG/the image as the actual fault.

Hi @Tom-Elliott
Here’s the table comparison you asked for:

Win10 (23H2), SSSTC drive:
label: gpt, protective MBR
p1: start=2048 size=204800 EF00 EFI
p2: start=206848 size=32768 0C01 MSR
p3: start=239616 size=498573824 (237.7 GiB) 0700 Basic Data
p4: start=498813440 size=1304576 (637.0 MiB) 2700 WinRE
Free: 2157 sectors (1.1 MiB), no problems found

Win11 (25H2), Toshiba drive:
label: gpt, protective MBR
p1: start=2048 size=204800 EF00 EFI
p2: start=206848 size=32768 0C01 MSR
p3: start=239616 size=498344448 (237.6 GiB) 0700 Basic Data
p4: start=498584064 size=1533952 (749.0 MiB) 2700 WinRE
Free: 2157 sectors (1.1 MiB), no problems found

Type GUIDs, attributes, partition order, and start sectors for p1/p2/p3 are identical between both. The only difference is where the p3/p4 boundary falls — Win11’s WinRE partition is ~112MB larger, which just shifts that one boundary. No extra/hidden partitions, no reordering, no odd attributes on either side, no MBR anomaly (both show “protective” cleanly). I don’t see anything in the table itself that looks like a red flag.

Given how clean this diff is, I’m not expecting the bisection’s first step (sgdisk -Z, table-only) to change anything, but running it now anyway as instructed. Will report whether F2/Setup opens after that, and if not, move to zeroing the ESP start next.

Thanks

Hi all, @Tom-Elliott
Update, and I want to push back on the VBS theory — I don’t think that’s actually it, despite what looked like supporting evidence.

New, cleaner data point: with the deployed drive installed, the machine doesn’t just hang at the Dell logo during normal boot — it also hangs during “Preparing to enter BIOS Setup” when pressing F2. That’s firmware enumerating storage before Setup even opens, with no OS/bootloader/VBS involved at that stage. I confirmed this persists even after a full CMOS battery removal (physically pulled the coin cell, left it disconnected several minutes, reseated) — so NVRAM/CMOS state is ruled out too.

The clean isolation test: same physical NVMe drive, same machine —

• Drive wiped (sgdisk -Z + dd zero on first/last 10MB) with nothing deployed → boots fine, F2 reaches Setup normally.
• Drive with the Windows 11 image deployed (via FOG, resizable or not) → both normal boot AND F2/Setup entry hang.
• Drive with a Windows 10 image deployed via the identical FOG/Hyper-V capture pipeline → boots fine, every time, F2 works.

So it’s specific to whatever ends up on the drive after a Windows 11 deploy — and since F2/Setup-entry is affected too, this is firmware choking while scanning the disk itself, not a Windows boot-chain issue. I’d chased VBS/HVCI/TPM for a while (disabling the master switch reduced but did not eliminate the intermittent hang — went from failing most of the time to failing ~1 in 6-10 boots, which in hindsight might just be noise/coincidence rather than a real effect from that change).

Given F2 itself is affected, I think this points back toward something written to the disk outside the partitions FOG restores — back to your original stale-metadata theory, just not RAID/mdadm specifically. Could this be something IMSM/Intel RST related that mdadm --examine doesn’t recognize, or a vendor-specific NVMe log/telemetry region that only Windows 11’s install process (vs. 10’s) writes to during setup? Genuinely unsure what else to check here — happy to run any other read-only diagnostics (nvme-cli, smartctl, anything) against the raw device to compare a “good” (Win10 or wiped) drive against a “bad” (Win11) one, sector-region by sector-region if needed.

TThanks

Update: ran the wipe-then-deploy sequence you suggested.
@Tom-Elliott
sgdisk -Z /dev/nvme0n1 → “GPT data structures destroyed”
dd zero on first 10MB and last 10MB of the disk (wipefs wasn’t available in this FOS build, used dd instead)
gdisk -l afterward confirmed: MBR/BSD/APM/GPT all “not present” — fully blank disk
mdadm --examine /dev/nvme0n1 → “No md superblock detected” (both before and after the wipe, for what it’s worth)

Then ran a normal Deploy task on this fully wiped disk. Result: same exact symptom. Stuck on Dell logo, unresponsive, after reboot.

I also manually selected “Boot from hard disk” from the FOG iPXE menu (instead of letting it fall through automatically) to rule out a boot-order/handoff issue — same hang.

So the stale-RAID-metadata-at-the-tail theory doesn’t hold up in this case, at least not as the sole cause — a completely zeroed disk with a fresh deploy still hangs. Back to square one on root cause, but happy to keep digging with more diagnostics if you have other ideas.

Thanks for the detailed breakdown — this is genuinely helpful, and I have some important corrections/clarifications after re-testing more carefully.

Terminology correction first: it’s not truly “hung.” The system stays responsive at the EC/keyboard level — pressing Caps Lock toggles the LED on and off reliably. So the machine is alive; it’s the display/firmware boot progression specifically that never advances past the Dell logo. F12 does not bring up the boot menu — the screen just stays on the logo, same as a normal boot attempt.

Critical new finding: this only happens after a FOG deploy. It is not tied merely to the drive’s physical presence.

To test this cleanly, I removed the NVMe, wiped/formatted it, and installed Windows fresh from a USB installer (not via FOG at all) onto the exact same physical drive, in the exact same machine. Result: it boots completely normally, no issue whatsoever, every time. F12 works, BIOS is reachable, Windows boots to desktop with no hang.

I then took that same drive and deployed the FOG image onto it (same image, same task). Result: back to the exact same symptom — stuck on the Dell logo, F12 unresponsive, indefinitely, on the exact same physical drive that had just booted a clean Windows install fine minutes earlier.

So the variable isn’t the drive, the brand, or even “a drive being present at POST” in general — it’s specifically whatever the FOG deploy leaves on that drive, compared to what a normal Windows Setup install leaves. Since Windows Setup typically converts/cleans the disk when you delete existing partitions and create new ones, while FOG’s resizable Partclone restore only writes the source partitions and leaves whatever sits outside those partition boundaries untouched — this seems to line up exactly with your stale-metadata-at-the-tail-of-the-drive theory. A regular USB Windows install would wipe that tail area as a side effect of repartitioning; a FOG resizable restore would not.

I’ll still run the diagnostic commands (wipefs, mdadm --examine, sgdisk -v, gdisk -l) in a Debug deploy task right after a deploy that reproduces the hang, and report back with the output, plus test your wipe-then-deploy sequence directly. I’ll also clear NVRAM and adjust the two BIOS settings you mentioned before retesting.

On /dev/md0: I didn’t set it manually, so if it’s not a FOG-side artifact, my best guess is it’s coming from something left on these drives before they reached me (factory RAID/IMSM config, or a prior imaging tool) — consistent with what you’re describing.

Will follow up with full command output shortly.

Hi all, @Tom-Elliott @Sebastian-Roth

I’m running into a hardware-agnostic boot failure after deploying a Windows 11 Enterprise image via FOG 1.6.0-beta.5384, and I’ve run out of ideas after fairly extensive diagnosis. Looking for insight from anyone who’s hit something similar.

Setup:

• Image: Windows 11 Enterprise, captured from a VM (not physical hardware)
• FOG Image Type: Single Disk - Resizable, Partclone Zstd
• Target: Dell Latitude 3400 — reproducible on 3 separate identical units

Symptom:
After a successful-looking deploy (Partclone completes 100%, UUIDs reset, “Task Complete”), the machine reboots and hangs indefinitely on the Dell logo. It does not progress, does not BSOD, and does not trigger Windows Automatic Repair even after 3+ forced power cycles — suggesting the hang happens very early, likely before Windows Boot Manager fully hands off to the OS.

What I’ve ruled out so far:

1. GPT/partition structure — Confirmed via gdisk/fdisk in FOG Debug mode that the disk has all 4 expected partitions (EFI System 100M, MSR 16M, NTFS Windows, WinRE 749M) after correcting the host’s “Host Primary Disk” field (it had auto-populated as /dev/md0 instead of /dev/nvme0n1 — possibly from Quick Registration).
2. Bootloader presence — Confirmed bootmgfw.efi exists and is intact in /EFI/Microsoft/Boot/ on the ESP.
3. BIOS storage mode — Already set to AHCI (not RAID) in BIOS.
4. SSD firmware bug — Initially suspected the SSSTC CL1-3D256-Q11 (firmware 22301113) had the known APST/low-power hang bug (Dell KB references firmware 22301114 fixing exactly this). However, swapping to a different brand (Toshiba) NVMe with a fresh deploy reproduces the exact same hang — ruling out a drive-specific firmware issue.
5. Hardware fault — Ruled out via testing: with no NVMe installed, the machine boots normally into BIOS/SupportAssist. The hang only occurs when an NVMe with this deployed image is present, regardless of brand.
6. Comparison: the exact same image deploys and boots perfectly on an HP ProOne 440 G9 AIO — so the image itself works, just not on this Dell model.

Current working theory:
Since the image was captured from a VM, I suspect either:

• Missing Dell-specific chipset/platform drivers causing an unrecoverable hang during hardware enumeration (rather than a slow PnP install)
• A TPM/VBS/BitLocker measured-boot mismatch — the VM likely used a virtual/absent TPM, and re-validation against the Latitude’s physical TPM on first physical boot might hang rather than gracefully fail
• Something firmware-specific to the Latitude 3400 UEFI implementation

Question for the community:
Has anyone seen FOG-deployed VM-captured images hang like this specifically on Dell Latitude hardware? Is there a recommended way to sanitize a VM-captured image (TPM/VBS state, generalize pass) before deploying to physical Dell hardware via FOG, short of re-capturing from physical hardware?

Any pointers appreciated — happy to provide more debug output if useful.

Environment:

• FOG Version: 1.6.0-beta.5384 (Channel Beta)
• OS: Ubuntu 26.04.1 LTS “Resolute”
• Target: Dell Latitude 3400

Thanks

Hi all, @Tom-Elliott @Sebastian-Roth

I found a bug in the Windows product key validation introduced in the Channel Beta | Version 1.6.0-beta.5384.

Issue:
A legitimate corporate Windows 11 Enterprise volume-license product key (containing the letter N) was rejected on the Host Management “General” tab with the error “Invalid Windows product key” (HTTP 400). The same key was accepted without issue on older FOG versions (pre-1.6.0-beta strict validation).

Root cause:
File: src/Base/FOGBase.php, function productKeyIsValid() (around line 2758)

public static function productKeyIsValid($val)
{
    return (bool)preg_match(
        '/^[BCDFGHJKMPQRTVWXY2346789]{25}$/',
        self::productKeyStrip($val)
    );
}

The allowed character class BCDFGHJKMPQRTVWXY2346789 is missing the letter N, which IS a valid character in the official Windows product key alphabet.

Evidence N is valid:
Many official Microsoft KMS Client Setup Keys contain N, e.g.:

• Windows 10/11 Pro: W269N-WFGWX-YVC9B-4J6C9-T83GX
• Windows 10/11 Enterprise: NPPR9-FWDCX-D2C8J-H872K-2YT43
• Windows 10/11 Pro N: MH37W-N47XK-V7XM9-C7227-GCQG9

Source: https://learn.microsoft.com/en-us/windows-server/get-started/kmsclientkeys

Steps to reproduce:

1. Host Management → edit a host → General tab
2. Enter a valid Windows Enterprise/volume product key containing the letter N
3. Click Update
4. FOG rejects it as “Invalid Windows product key”

Suggested fix:
Add N to the character class:
‘/^[BCDFGHJKMNPQRTVWXY2346789]{25}$/’

Workaround applied locally (via sed) while waiting for an official fix:

sudo sed -i “s/[BCDFGHJKMPQRTVWXY2346789]/[BCDFGHJKMNPQRTVWXY2346789]/” /var/www/fog/src/Base/FOGBase.php

Confirmed working after this change.

Environment:

• FOG Version: 1.6.0-beta.5384 (Channel Beta, branch working-1.6)
• OS: Ubuntu 26.04.1 LTS “Resolute”

Thanks!