• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    [BUG] iPXE Boot Loop & Menu Failure After FOG 1.5.10.1660 Upgrade

    Scheduled Pinned Locked Moved Unsolved
    FOG Problems
    2
    16
    186
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J
      Jim Holcomb
      last edited by

      [BUG] iPXE Boot Loop and Kernel Missing After Upgrade to FOG 1.5.10.1660 — Clients Fail to Boot to Menu

      Environment:

      • FOG Version: 1.5.10.1660
      • Server IP: 192.168.9.10
      • Clients: Lenovo M625Q Tiny, UEFI BIOS
      • Primary Disk: 256GB SSD
      • Secondary Disk: 500GB HDD
      • PXE Boot Mode: UEFI
      • Install Directory: /root/fogproject/
      • Apache DocRoot: /var/www/

      Issue Summary:

      After upgrading to the latest FOG version, my PXE boot environment is broken. What used to work flawlessly is now a disaster. PXE clients either hang, enter a loop, or display autoexec.ipxe not found. Sometimes pressing ESC brings up a green DOS-style login prompt, then fails to chainload. Sometimes they hang on “Booting…” indefinitely.

      Symptoms:

      • iPXE boot screen says:
        • Could not boot: autoexec.ipxe not found
        • Chainloading failed
      • Green DOS login prompt appears briefly after pressing ESC
      • Clients never reach the expected FOG PXE Menu
      • Kernel files (bzImage, bzImage32) and init files are missing
      • FOG configuration GUI shows blank or missing sections
      • No active task, yet PXE fails to display menu
      • boot.php loads in browser but returns:
        Warning: A non-numeric value encountered in bootmenu.class.php on line 290

      What I’ve Verified:

      • Tables hosts, hostMAC, and ipxeMenuItems are populated and correct
      • MAC address correctly mapped to host
      • PXE config references bzImage32, but the file is missing
      • Apache serving from /var/www, not /var/www/html
      • Manually created /var/www/autoexec.ipxe, permissions confirmed
      • Kernel paths match database and boot.php but files are gone
      • curl test against boot.php returns valid iPXE script, but boot fails
      • No active tasks found in the tasks table

      What’s Missing or Broken:

      • /var/www/fog/service/ipxe/ is missing bzImage, init.xz, etc.
      • GUI options like FOG Configuration → iPXE Menu Configuration are missing
      • No visible way to manage PXE boot menu via GUI
      • Default behavior not working: No task = no PXE menu shown

      What I’ve Tried:

      • Rebuilt PXE menu records in MySQL
      • Manually added menuaccess=1 in boot.php call to test
      • Disabled display_errors in PHP FPM, CLI, and Apache configs
      • Reinstalled Apache/PHP, confirmed docroot
      • Cleared and reloaded /tftpboot
      • Re-created /var/www/autoexec.ipxe
      • Spent over 14 man-hours debugging this
      • Absolutely no unauthorized edits to boot.php

      This Is Critical:

      FOG’s original behavior:

      • If task is scheduled → auto-execute
      • If no task → show PXE boot menu

      That logic is now broken post-upgrade. I do not want custom behavior or rewritten logic. Just default.

      Ask to Community:

      • Is this a known issue in 1.5.10.1660?
      • Where can I safely re-download the kernel/init files (like bzImage32, init.xz)?
      • How do I regenerate the boot.php environment cleanly without destroying host/task data?
      • Can anyone else confirm GUI menus missing in this build?

      Attachments Available:

      • Screenshots of:
      • PXE Boot failures
      • Green login prompt
      • Missing menu entries
      • SQL output verifying host mapping

      Thanks in advance — desperate for resolution.

      – Jim

      Tom ElliottT 2 Replies Last reply Reply Quote 0
      • Tom ElliottT
        Tom Elliott @Jim Holcomb
        last edited by

        @Jim-Holcomb If you rerun the fog installer, this should fix your boot.php and reaches out to download and place the bzImage(s) and init(s) files.

        Please re-run the installer, and anything that is currently broken from customization should at least be set back to what was produced for the 1660 information.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

        J 1 Reply Last reply Reply Quote 0
        • J
          Jim Holcomb @Tom Elliott
          last edited by

          @Tom-Elliott
          Thanks Tom

          However I ran this about 15 times. Post install I was never given an option to update the database or schema.

          My initial issue before upgrading was that I was trying to deploy image to Lenovo M625Q tiny with 256gb SSD drive as the PRIMARY drive and the standard Seagate 500gb (SATA0) as the SECONDARY drive. I added the /dev/nvme0n1 as the primary disk, yet FoG wrote to the Seagate drive first every time.

          I asked ChatGPT for ideas, and it found a known issue with FoG writing to EUFI SSD drive as Primary drive and stated that I needed to update to the latest iPXE file. Everything been broken ever since. I have spent more than a week trying to get FoG working once again.

          I’m updating my Ubuntu server to v20.04 now and I will try the re-install one more time, unless you have any other ideas?

          -Jim

          Tom ElliottT 1 Reply Last reply Reply Quote 0
          • Tom ElliottT
            Tom Elliott @Jim Holcomb
            last edited by

            @Jim-Holcomb if the boot.php was updated from the install folder, then you’d like need to repull the repository.

            This can be done by going to your installer path. running:

            git reset --hard
git clean -xfd
git checkout stable
cd bin
./installfog.sh -y

            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

            Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

            Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

            J 1 Reply Last reply Reply Quote 0
            • J
              Jim Holcomb @Tom Elliott
              last edited by

              @Tom-Elliott said in [BUG] iPXE Boot Loop & Menu Failure After FOG 1.5.10.1660 Upgrade:

              git reset --hard
              git clean -xfd
              git checkout stable
              cd bin
              ./installfog.sh -y

              Tom, I did this again (16th time now) and I never get prompted to update database or schema. Is this normal?

              I still can’t get past the iPXE (v1.21.1+) screen!iPXE121.jpg

              Tom ElliottT 2 Replies Last reply Reply Quote 0
              • J
                Jim Holcomb
                last edited by

                Now this:

                ChatGPT says: “Your system booted in UEFI mode, but bzImage32 is a 32-bit kernel, which cannot boot on most 64-bit UEFI systems.”

                Does this help?

                -JimiPXE121-2.jpg

                1 Reply Last reply Reply Quote 0
                • Tom ElliottT
                  Tom Elliott @Jim Holcomb
                  last edited by

                  @Jim-Holcomb Not all updates will force a db update. That is normal.

                  Looking at the error in question now that it’s been shown more clearly.

                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                  Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                  Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                  1 Reply Last reply Reply Quote 0
                  • Tom ElliottT
                    Tom Elliott @Jim Holcomb
                    last edited by

                    @Jim-Holcomb I’m seeing you have boot access enabled?

                    I don’t know why bzImage32 is being loaded.

                    Seems to me you likely need to check the host or global kernel settings. 64 bit kernel should be defined to bzImage This would show up under global configurations (FOG COnfiguration Page -> FOG Settings) “TFTP_PXE_KERNEL” should be set to bzImage
                    TFTP_PXE_KERNEL_32 should be set to bzImage32

                    THe other place to test would be under the host directly -> Host Kernel.

                    Please check that those are set appropriately.

                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                    1 Reply Last reply Reply Quote 0
                    • J
                      Jim Holcomb
                      last edited by

                      @Tom-Elliott said in [BUG] iPXE Boot Loop & Menu Failure After FOG 1.5.10.1660 Upgrade:

                      Seems to me you likely need to check the host or global kernel settings. 64 bit kernel should be defined to bzImage This would show up under global configurations (FOG COnfiguration Page -> FOG Settings) “TFTP_PXE_KERNEL” should be set to bzImage
                      TFTP_PXE_KERNEL_32 should be set to bzImage32

                      Tom, where can I find these settings? I am in Fog Settings, but I only see lot of blue links, none of which refer to Kernel or TFTP?
                      -J

                      1 Reply Last reply Reply Quote 0
                      • J
                        Jim Holcomb
                        last edited by

                        @Tom-Elliott said in [BUG] iPXE Boot Loop & Menu Failure After FOG 1.5.10.1660 Upgrade:

                        Seems to me you likely need to check the host or global kernel settings. 64 bit kernel should be defined to bzImage This would show up under global configurations (FOG COnfiguration Page -> FOG Settings) “TFTP_PXE_KERNEL” should be set to bzImage
                        TFTP_PXE_KERNEL_32 should be set to bzImage32

                        These are set correctly
                        -J

                        Tom ElliottT 1 Reply Last reply Reply Quote 0
                        • Tom ElliottT
                          Tom Elliott @Jim Holcomb
                          last edited by

                          @Jim-Holcomb Is it just this one machine having this problem or ALL your machines are using bzImage32?

                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                          1 Reply Last reply Reply Quote 0
                          • J
                            Jim Holcomb
                            last edited by

                            FoG was running perfectly for years, as you may recall. The only recent changes are that we are no longer using Legacy Bios for the imaging - as we have added a secondary SSD drive (M.2 on SATA1) to the equation. However FoG was only sending the Primary image to SATA0 no matter what I put in FoG. However these same machines with a single standard Seagate 500gb drive in SATA0 was working fine under EUFI, both DEPLOY and CAPTURE. Ai recommended a change in the iPXE file, which started this whole nonsense. So this time we wiped the pxe files from TFTP and did a reinstall of FoG. Getting closer now, but still getting the same errors on the PXE screen.

                            Pretty much at my wit’s end here. I know were are close…

                            -J

                            1 Reply Last reply Reply Quote 0
                            • J
                              Jim Holcomb
                              last edited by

                              Tom, I am pretty sure this is a kernel issue - but this is beyond my knowledge. Are you available to assist remotely?

                              -Jim

                              1 Reply Last reply Reply Quote 0
                              • J
                                Jim Holcomb
                                last edited by

                                Tom I was able to get everything back up and running with the assistance of ChatGPT - but it only took a little over a week! (And for anyone else reading this, I would highly NOT recommend using ChatGPT for FoG assistance)

                                However Tom I am stuck with the same issue. FoG will not write to the correct partitions. Once again here is my setup:

                                • Lenovo ThinkCentre M615Q tiny;

                                  32GB RAM
                                  500gb Seagate HDD in SATA0
                                  256gb SSD drive (M.2) in SATA1 (these are not cabled, not changeable)
                                  NOTE: I setup the PC with the SSD drive being the main PRIMARY drive (faster speed, “C” drive) and the Seagate standard HDD as the SECONDARY drive (“D” drive)

                                The device was imaged as a “Multiple Partition Image - Single Disk (not resizable) - (2)” as the SSD drive has the standard 3 or 4 partitions that Windows setups up by default.
                                Host was setup using /dev/nvme0n1 as the “Host Primary Disk”. It was captured without error. I verified via diskpart before imaging that each HDD/SDD was GPT partitions. However when I do a file listing of the image, I am seeing a d1.mbr file. Is this correct?
                                1024 -rwxrwxr-x 1 fogproject fogproject 1048576 Jun 7 16:21 d1.mbr
                                13268 -rwxrwxr-x 1 fogproject fogproject 13584339 Jun 7 16:21 d1p1.img
                                4576 -rwxrwxr-x 1 fogproject fogproject 4684567 Jun 7 16:21 d1p2.img
                                18382576 -rwxrwxr-x 1 fogproject fogproject 18823753628 Jun 7 16:49 d1p3.img
                                454724 -rwxrwxr-x 1 fogproject fogproject 465637248 Jun 7 16:50 d1p4.img
                                4 -rwxrwxr-x 1 fogproject fogproject 873 Jun 7 16:21 d1.partitions

                                When I watch the DEPLOY task, it says it’s writing to the /dev/nvme0n1 disk, but when the deploy is completed, I can verify that it wrote to both drives, but not in order.

                                Can you help me with this PLEASE?

                                -Jim

                                1 Reply Last reply Reply Quote 0
                                • J
                                  Jim Holcomb
                                  last edited by

                                  Also, digging deeper, it appears as though it is a good GPT image file:
                                  cat /images/EUFI2HDD2025x/d1.partitions
                                  label: gpt
                                  label-id: 90AAA622-0DBD-4BE4-9D85-A54D50340EC6
                                  device: /dev/nvme0n1
                                  unit: sectors
                                  first-lba: 34
                                  last-lba: 500118158
                                  sector-size: 512

                                  /dev/nvme0n1p1 : start= 2048, size= 204800, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=8E1F3336-F9E2-4C7F-A832-B8865182DF2D, name=“EFI system partition”, attrs=“GUID:63”
                                  /dev/nvme0n1p2 : start= 206848, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=BAEB99F3-C8AC-4FAE-8CC7-95C7D412A219, name=“Microsoft reserved partition”, attrs=“GUID:63”
                                  /dev/nvme0n1p3 : start= 239616, size= 498755998, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=B16B4307-0A5C-42AD-B6EA-8A783ABC4C8C, name=“Basic data partition”
                                  /dev/nvme0n1p4 : start= 498997248, size= 1118208, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=DDE992F8-5913-46DB-8E25-B0AD6C84266A, attrs=“RequiredPartition GUID:63”

                                  -Jim

                                  1 Reply Last reply Reply Quote 0
                                  • Tom ElliottT
                                    Tom Elliott @Jim Holcomb
                                    last edited by

                                    @Jim-Holcomb So unfortunately i’m not readily available for a remote assist due to work requirements and duties so I apologize.

                                    @Jim-Holcomb said in [BUG] iPXE Boot Loop & Menu Failure After FOG 1.5.10.1660 Upgrade:

                                    After upgrading to the latest FOG version, my PXE boot environment is broken. What used to work flawlessly is now a disaster. PXE clients either hang, enter a loop, or display autoexec.ipxe not found. Sometimes pressing ESC brings up a green DOS-style login prompt, then fails to chainload. Sometimes they hang on “Booting…” indefinitely.

                                    So this “autoexec.ipxe not found” always shows. This is not an issue. It’s simply saying there’s no file called autoexec.ipxe on your /tftpboot folder. It’s NOT whatever was causing ipxe boot issues. I’m guessing this PXE boot environment was broken due to whatever was changed?

                                    As to the selective hard drive. I’m not sure I understand what is wrong here. YOu watched it write ONLY to the /dev/nvme drive but somehow it wrote to both the sda and the nvme? Something doesn’t sound right there. Single Disk resizable and Multiple Partition Single disk shoudl only write to a single drive and be done. Do you have video of this process writing to both? I only ask because seeing this in action will help me understand where the code is flawed. To my knowledge, however, it seems more likely that at some point before you wrote to the NVME it had accidentally written to the SDA drive, then you try to figure out how to have it write to the NVME. Then, finally, it was able to write to the NVME, but now it LOOKS like it wrote to both because the SDA drive was never cleared out?

                                    And yes, d1.mbr is absolutely correct. That’s how it initializes the disk.

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                    1 Reply Last reply Reply Quote 0
                                    • 1 / 1
                                    • First post
                                      Last post

                                    213

                                    Online

                                    12.1k

                                    Users

                                    17.3k

                                    Topics

                                    155.3k

                                    Posts
                                    Copyright © 2012-2024 FOG Project