• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Windows 11 image captured from VM hangs indefinitely at Dell logo on Latitude 3400 (multiple units, multiple NVMe brands) — works fine on HP

    Scheduled Pinned Locked Moved General
    7 Posts 2 Posters 53 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      servicedesk.pianezza
      last edited by

      Hi all, @Tom-Elliott @Sebastian-Roth

      I’m running into a hardware-agnostic boot failure after deploying a Windows 11 Enterprise image via FOG 1.6.0-beta.5384, and I’ve run out of ideas after fairly extensive diagnosis. Looking for insight from anyone who’s hit something similar.

      Setup:

      • Image: Windows 11 Enterprise, captured from a VM (not physical hardware)
      • FOG Image Type: Single Disk - Resizable, Partclone Zstd
      • Target: Dell Latitude 3400 — reproducible on 3 separate identical units

      Symptom:
      After a successful-looking deploy (Partclone completes 100%, UUIDs reset, “Task Complete”), the machine reboots and hangs indefinitely on the Dell logo. It does not progress, does not BSOD, and does not trigger Windows Automatic Repair even after 3+ forced power cycles — suggesting the hang happens very early, likely before Windows Boot Manager fully hands off to the OS.

      What I’ve ruled out so far:

      1. GPT/partition structure — Confirmed via gdisk/fdisk in FOG Debug mode that the disk has all 4 expected partitions (EFI System 100M, MSR 16M, NTFS Windows, WinRE 749M) after correcting the host’s “Host Primary Disk” field (it had auto-populated as /dev/md0 instead of /dev/nvme0n1 — possibly from Quick Registration).
      2. Bootloader presence — Confirmed bootmgfw.efi exists and is intact in /EFI/Microsoft/Boot/ on the ESP.
      3. BIOS storage mode — Already set to AHCI (not RAID) in BIOS.
      4. SSD firmware bug — Initially suspected the SSSTC CL1-3D256-Q11 (firmware 22301113) had the known APST/low-power hang bug (Dell KB references firmware 22301114 fixing exactly this). However, swapping to a different brand (Toshiba) NVMe with a fresh deploy reproduces the exact same hang — ruling out a drive-specific firmware issue.
      5. Hardware fault — Ruled out via testing: with no NVMe installed, the machine boots normally into BIOS/SupportAssist. The hang only occurs when an NVMe with this deployed image is present, regardless of brand.
      6. Comparison: the exact same image deploys and boots perfectly on an HP ProOne 440 G9 AIO — so the image itself works, just not on this Dell model.

      Current working theory:
      Since the image was captured from a VM, I suspect either:

      • Missing Dell-specific chipset/platform drivers causing an unrecoverable hang during hardware enumeration (rather than a slow PnP install)
      • A TPM/VBS/BitLocker measured-boot mismatch — the VM likely used a virtual/absent TPM, and re-validation against the Latitude’s physical TPM on first physical boot might hang rather than gracefully fail
      • Something firmware-specific to the Latitude 3400 UEFI implementation

      Question for the community:
      Has anyone seen FOG-deployed VM-captured images hang like this specifically on Dell Latitude hardware? Is there a recommended way to sanitize a VM-captured image (TPM/VBS state, generalize pass) before deploying to physical Dell hardware via FOG, short of re-capturing from physical hardware?

      Any pointers appreciated — happy to provide more debug output if useful.

      Environment:

      • FOG Version: 1.6.0-beta.5384 (Channel Beta)
      • OS: Ubuntu 26.04.1 LTS “Resolute”
      • Target: Dell Latitude 3400

      Thanks

      Tom ElliottT 1 Reply Last reply Reply Quote 0
      • Tom ElliottT
        Tom Elliott @servicedesk.pianezza
        last edited by

        @servicedesk-pianezza Thanks for the detail — that is a good writeup, and it saves a lot of back and forth.

        Two things before the questions.

        First, one thing you can rule out on our side: FOS never writes UEFI boot entries. There is
        no efibootmgr anywhere in the FOS image, so a deploy cannot leave a bad NVRAM entry
        behind. Nothing on the server writes the Host Primary Disk field either — not full
        registration, not Quick Registration, not inventory. The only writers are the host edit
        form, the group form and the API. So the /dev/md0 you found was set by a person or by an
        import, and it is a clue, not a side effect: /dev/md0 can only appear in FOS if an md
        array was assembled, and FOS only does that when the host boots with the mdraid=true
        kernel argument. That means one of those disks carried RAID (or Intel RST / IMSM) metadata.

        Second, your symptom has been reported twice before on this exact family and neither
        report reached a root cause:

        • https://forums.fogproject.org/topic/14191 — Latitude 3400, resizable, stuck at the Dell
          splash, “the only fix is to wipe the drive and re-format it”
        • https://forums.fogproject.org/topic/14147 — Latitude 3500, resizable, same, and the same
          cure

        In both, the drive booted fine once moved to another Dell model, and only a wipe on another
        machine made the 3400/3500 usable again. That points at the firmware stalling while it
        scans the disk, not at Windows. A wipe removes more than the partition table — it removes
        whatever is left outside the partitions FOG restores.

        So the working theory is stale RAID metadata in the tail of the drive. A resizable deploy
        writes the partition table and the partitions; it does not zero the rest of the disk, so
        anything the factory install left at the end of the drive survives. Your /dev/md0 is
        direct evidence that metadata was there.

        Could you run these? All of them in a debug deploy task (tick Debug on the task), at the
        shell, before you type fog:

        wipefs /dev/nvme0n1                 # lists signatures, changes nothing
        mdadm --examine /dev/nvme0n1        # and on each partition, e.g. nvme0n1p4
        sgdisk -v /dev/nvme0n1
        gdisk -l /dev/nvme0n1               # the header lines, including any warnings
        

        Then, on a machine that is already hanging, the one test that separates firmware from
        Windows: with the deployed drive in, does F12 reach the boot menu, or does the machine hang
        before that too? And with the drive wiped (sgdisk -Z /dev/nvme0n1; wipefs -a /dev/nvme0n1) but nothing deployed, does it boot to “no bootable device”?

        If the wipe-then-deploy sequence boots, that is the answer and we will make FOG clear those
        signatures itself.

        Two BIOS settings worth clearing on these while you are in there, both known to cause a
        logo hang independent of FOG: set SupportAssist OS Recovery / “Auto OS Recovery Threshold”
        to off, and set Fastboot to Thorough. Also clear NVRAM once, since a 3400 that has failed
        to boot several times will have accumulated stale boot entries.

        On your sanitize question: a VM capture needs sysprep /generalize /oobe /shutdown before
        the capture, and that is the whole of it for hardware differences. It does not explain a
        hang this early — generalize problems show up as a BSOD or a spinner, not as a freeze at
        the vendor logo.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

        1 Reply Last reply Reply Quote 0
        • S
          servicedesk.pianezza
          last edited by

          Thanks for the detailed breakdown — this is genuinely helpful, and I have some important corrections/clarifications after re-testing more carefully.

          Terminology correction first: it’s not truly “hung.” The system stays responsive at the EC/keyboard level — pressing Caps Lock toggles the LED on and off reliably. So the machine is alive; it’s the display/firmware boot progression specifically that never advances past the Dell logo. F12 does not bring up the boot menu — the screen just stays on the logo, same as a normal boot attempt.

          Critical new finding: this only happens after a FOG deploy. It is not tied merely to the drive’s physical presence.

          To test this cleanly, I removed the NVMe, wiped/formatted it, and installed Windows fresh from a USB installer (not via FOG at all) onto the exact same physical drive, in the exact same machine. Result: it boots completely normally, no issue whatsoever, every time. F12 works, BIOS is reachable, Windows boots to desktop with no hang.

          I then took that same drive and deployed the FOG image onto it (same image, same task). Result: back to the exact same symptom — stuck on the Dell logo, F12 unresponsive, indefinitely, on the exact same physical drive that had just booted a clean Windows install fine minutes earlier.

          So the variable isn’t the drive, the brand, or even “a drive being present at POST” in general — it’s specifically whatever the FOG deploy leaves on that drive, compared to what a normal Windows Setup install leaves. Since Windows Setup typically converts/cleans the disk when you delete existing partitions and create new ones, while FOG’s resizable Partclone restore only writes the source partitions and leaves whatever sits outside those partition boundaries untouched — this seems to line up exactly with your stale-metadata-at-the-tail-of-the-drive theory. A regular USB Windows install would wipe that tail area as a side effect of repartitioning; a FOG resizable restore would not.

          I’ll still run the diagnostic commands (wipefs, mdadm --examine, sgdisk -v, gdisk -l) in a Debug deploy task right after a deploy that reproduces the hang, and report back with the output, plus test your wipe-then-deploy sequence directly. I’ll also clear NVRAM and adjust the two BIOS settings you mentioned before retesting.

          On /dev/md0: I didn’t set it manually, so if it’s not a FOG-side artifact, my best guess is it’s coming from something left on these drives before they reached me (factory RAID/IMSM config, or a prior imaging tool) — consistent with what you’re describing.

          Will follow up with full command output shortly.

          1 Reply Last reply Reply Quote 0
          • S
            servicedesk.pianezza
            last edited by

            Update: ran the wipe-then-deploy sequence you suggested.
            @Tom-Elliott
            sgdisk -Z /dev/nvme0n1 → “GPT data structures destroyed”
            dd zero on first 10MB and last 10MB of the disk (wipefs wasn’t available in this FOS build, used dd instead)
            gdisk -l afterward confirmed: MBR/BSD/APM/GPT all “not present” — fully blank disk
            mdadm --examine /dev/nvme0n1 → “No md superblock detected” (both before and after the wipe, for what it’s worth)

            Then ran a normal Deploy task on this fully wiped disk. Result: same exact symptom. Stuck on Dell logo, unresponsive, after reboot.

            I also manually selected “Boot from hard disk” from the FOG iPXE menu (instead of letting it fall through automatically) to rule out a boot-order/handoff issue — same hang.

            So the stale-RAID-metadata-at-the-tail theory doesn’t hold up in this case, at least not as the sole cause — a completely zeroed disk with a fresh deploy still hangs. Back to square one on root cause, but happy to keep digging with more diagnostics if you have other ideas.

            Tom ElliottT 1 Reply Last reply Reply Quote 0
            • Tom ElliottT
              Tom Elliott @servicedesk.pianezza
              last edited by

              @servicedesk-pianezza Those are clean tests and they kill my theory. Correcting the record, and then I think your
              own last test moves this a long way.

              The stale-metadata idea is dead. A zeroed disk with a fresh deploy still hangs, and
              mdadm --examine found no superblock either side of the wipe. Drop it.

              I also replayed our GPT restore path locally against a Windows 11 resizable image, onto a
              disk deliberately smaller than the captured one — the same order FOS uses: dd of d1.mbr,
              sgdisk -z, sgdisk -gl, then the filldisk table through sfdisk. The result verifies clean:
              sgdisk -v reports no problems, the protective MBR is a single 0xEE entry spanning the whole
              device, and first/last usable sectors match the target. So a malformed partition table is not
              what we are looking at either.

              Now the part I think you undersold. You reached the FOG iPXE menu and chose “Boot from hard
              disk”, with the deployed drive fitted, and then it hung. That means the firmware finished
              POST, brought up the NIC, ran iPXE and drew a menu — all with that drive present. The
              firmware is not the thing that hangs.
              It hands off to bootmgfw.efi and the hang is after
              that point.

              Which reframes the symptom. “Stuck at the Dell logo” is not the firmware stalling. It is the
              Windows boot chain hanging before anything repaints the screen, so the OEM logo simply stays
              up. F12 being dead is expected there — the firmware gave up the keyboard at handoff. It also
              explains the missing Automatic Repair: Windows’ boot-failure counter is incremented by the
              boot manager, and a hang never reaches the code that does it.

              So the question is now why this image hangs in early Windows boot on a Whiskey Lake Latitude
              and not on a 12th-gen HP. Two things to do, in this order.

              Deploy the same image to the same Dell as Single Disk (Not Resizable). This splits the
              problem in half for the cost of one deploy. Both 2020 reports on this hardware said
              non-resizable worked where resizable did not — see banana123 in topic 14147, “Using Multiple
              Partition Image - Single Disk (Not Resizable) DOES work fine”. If non-resizable boots, the
              fault is in our resize path, it is ours, and I will want d1.minimum.partitions,
              d1.fixed_size_partitions and a full debug-deploy transcript. If it hangs too, the resize
              path is exonerated and it is the image.

              Make Windows tell you where it stops. Boot a Windows installer USB on the hung machine,
              Shift-F10 for a command prompt, find the ESP letter with diskpart, then:

              bcdedit /store X:\EFI\Microsoft\Boot\BCD /set {default} sos on
              bcdedit /store X:\EFI\Microsoft\Boot\BCD /set {globalsettings} bootmenupolicy legacy
              

              sos replaces the logo with the list of boot drivers as they load, so the screen names the
              last thing it got to instead of showing you a logo. bootmenupolicy legacy gives you the F8
              menu, and Safe Mode is itself a useful result.

              Last question, because you have not said it anywhere in the thread: was the image captured
              after sysprep /generalize /oobe /shutdown, or from a VM that had simply been shut down? An
              image captured without generalize carries the source machine’s driver and device state, and
              booting on one chipset but not another is the usual way that shows up.

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

              1 Reply Last reply Reply Quote 0
              • S
                servicedesk.pianezza
                last edited by

                Hi all, @Tom-Elliott
                Update, and I want to push back on the VBS theory — I don’t think that’s actually it, despite what looked like supporting evidence.

                New, cleaner data point: with the deployed drive installed, the machine doesn’t just hang at the Dell logo during normal boot — it also hangs during “Preparing to enter BIOS Setup” when pressing F2. That’s firmware enumerating storage before Setup even opens, with no OS/bootloader/VBS involved at that stage. I confirmed this persists even after a full CMOS battery removal (physically pulled the coin cell, left it disconnected several minutes, reseated) — so NVRAM/CMOS state is ruled out too.

                The clean isolation test: same physical NVMe drive, same machine —

                • Drive wiped (sgdisk -Z + dd zero on first/last 10MB) with nothing deployed → boots fine, F2 reaches Setup normally.
                • Drive with the Windows 11 image deployed (via FOG, resizable or not) → both normal boot AND F2/Setup entry hang.
                • Drive with a Windows 10 image deployed via the identical FOG/Hyper-V capture pipeline → boots fine, every time, F2 works.

                So it’s specific to whatever ends up on the drive after a Windows 11 deploy — and since F2/Setup-entry is affected too, this is firmware choking while scanning the disk itself, not a Windows boot-chain issue. I’d chased VBS/HVCI/TPM for a while (disabling the master switch reduced but did not eliminate the intermittent hang — went from failing most of the time to failing ~1 in 6-10 boots, which in hindsight might just be noise/coincidence rather than a real effect from that change).

                Given F2 itself is affected, I think this points back toward something written to the disk outside the partitions FOG restores — back to your original stale-metadata theory, just not RAID/mdadm specifically. Could this be something IMSM/Intel RST related that mdadm --examine doesn’t recognize, or a vendor-specific NVMe log/telemetry region that only Windows 11’s install process (vs. 10’s) writes to during setup? Genuinely unsure what else to check here — happy to run any other read-only diagnostics (nvme-cli, smartctl, anything) against the raw device to compare a “good” (Win10 or wiped) drive against a “bad” (Win11) one, sector-region by sector-region if needed.

                TThanks

                Tom ElliottT 1 Reply Last reply Reply Quote 0
                • Tom ElliottT
                  Tom Elliott @servicedesk.pianezza
                  last edited by

                  @servicedesk-pianezza You are right and I was wrong. F2 hanging settles it. “Preparing to enter BIOS Setup” runs
                  before any bootloader, so the firmware is the thing that stalls, and my reading of the iPXE
                  test was bad. Ignore that whole post.

                  Your Windows 10 control is the best evidence in this thread. Same capture pipeline, same
                  FOG, same drive, same machine, and it boots. Together with “resizable and non-resizable both
                  hang”, that clears our resize path and it clears FOG’s table writing as a general fault: we
                  are laying down the captured disk faithfully, and this particular captured disk upsets this
                  particular firmware.

                  So the question is narrow now: which bytes? Two things, and the first one you can do from
                  your desk.

                  Diff the two tables. You have a good drive and a bad drive from the same pipeline. In a
                  debug task on each, capture:

                  sfdisk -d /dev/nvme0n1
                  sgdisk -v /dev/nvme0n1
                  gdisk -l /dev/nvme0n1
                  

                  Post both sets. Partition count, order, types, attributes and the end-of-disk figures are
                  the things the firmware reads at enumeration, and a diff of Win10-good against Win11-bad
                  names the difference without any guessing.

                  Then bisect the disk, in this order. Start from a deployed drive that reproduces the
                  hang, and after each step power off, power on, press F2, and note whether Setup opens.

                  Destroy the partition table only, leaving every byte of partition data where it lies:

                  sgdisk -Z /dev/nvme0n1
                  

                  If Setup now opens, the firmware is choking on the partition table — layout, types or
                  attributes — and not on anything inside the partitions. If it still hangs, the data is the
                  problem, so redeploy and zero the start of the ESP, which is the only partition the firmware
                  reads:

                  dd if=/dev/zero of=/dev/nvme0n1p1 bs=1M count=1
                  

                  If Setup opens after that, it is the ESP filesystem, and we can bisect it file by file from
                  there.

                  Three questions while you are at it. Which Windows 11 build is the image, and which build
                  was the Windows 10 one? Was the Hyper-V VM Generation 2 with Secure Boot and a vTPM
                  attached? And what BIOS version are the 3400s on — Dell’s last for that model is in the 1.3x
                  range, and firmware is the component under suspicion now.

                  Also worth saying plainly: if this turns out to be the Latitude 3400 firmware choking on a
                  partition layout it does not like, there may be nothing for FOG to fix beyond documenting
                  it. Two earlier reporters on this hardware ended up rebuilding the image from a clean install
                  rather than finding a cause. I would rather find it, and your Win10 control is the first
                  thing anyone has produced that makes that realistic.

                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                  Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                  Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                  1 Reply Last reply Reply Quote 0
                  • 1 / 1
                  • First post
                    Last post

                  50

                  Online

                  12.8k

                  Users

                  17.6k

                  Topics

                  157.1k

                  Posts
                  Copyright © 2012-2026 FOG Project