ESXi PXE Boot PSOD/Crash
-
CentOS 7.4.1708 base
FOG: 1.5.2
FOG server is only server on its own subnet acting as dhcp and pxe, http, tftp, etc.
Host: Dell PowerEdge R620Added a menu entry under iPXE for VMWare ESXi (6.5 U1). I have the contents of the cd/dvd in /images/desxi/
I have edited the boot.cfg in /images/desxi/efi/boot/ to have the prefix of my server path to the path of these files (http://ip/fog/images/desxi) and have removed all the leading ‘/’'s in the modules and in front of the tboot.b00.
I followed this post as a guide:
http://forum.ipxe.org/showthread.php?tid=9477&pid=14952#pid14952I have verbatim what they have in terms of my boot.cfg and my menu entry except I am using my paths.
For UEFI its a bit different (adjusting menu entry to use bootx64.efi):
It flashes that bootx64.efi loads/ok
I then get the VMWare ESXi screen loading all the files needed.
It gets to ESXi’s screen with the progress bar, but I get a PSOD (Pink/Purple Screen of Death) when it gets to storage-stack in that process. At this point a hard reboot is required.Some stuff I have tried:
- I have tried loading the bootx64.efi and the boot.cfg via http and tftp (either loads these 2 files)
- I have tried setting the boot.cfg prefix parameter to: http, tftp, nfs (nfs doesnt work, other 2 load all the files/modules listed).
In all cases no matter how I successfully load the bootx64.efi and boot.cfg file and no matter how I load the modules from the boot.cfg file, once I get past loading the modules in ESXi’s loading screen and get to the installer, it crashes right towards the start on loading storage-stack or something along those lines, showing a PSOD.
I have checked the forums and found a similar post but without resolution and tried altering my menu to match with similar results
-
PXE booting a VMWare virtual machine I can get into the installer and start to install. The install failed because I didn’t have enough vCPUs allocated to the VM to install ESXi inside a ESXi virtual machine.
On the second pass with 4 vCPUs added I had a warning about virtualization was not enabled in the firmware.
At this point it does look like the installer is functioning as intended.
-
@george1421 I am pretty sure this is an iPXE issue, I found this thread over at iPXE forums thats exactly my problem (on different hardware). As they describe it, its when they use iPXE with that hardware and ESXi that the problem surfaces.
They however do not seem to have found a solution, which is a problem for me as I need to serve up ESXi via PXE and will need to regularly on hardware. I had other issues using SYSLINUX instead for it, so I was really hoping an alternative to Syslinux would work.
Is it possible to boot from ISO via UEFI boot? I know I could do memdisk via BIOS, looking for something similar via UEFI as a possible alternative if this cant be worked out.
Thanks
-
@zer0cool I’m not sure I understand your issue. I can use a fog server to pxe boot into the ESXi 6.5u1 boot media. I do not get the PSOD unless its after the CPU check for virtualization. As far as I can tell it works in uefi mode. If you don’t want to use uefi mode then you need to compile your own iPXE boot loader (which is also possible) to enable the COMBOOT option which will give you the syslinux-like menu options.
Let me grab a Dell 7010 to see if I can get ESXi fully installed.
-
@george1421 Sorry I think we are on different pages here.
I DO need UEFI boot.
If you UEFI booted on hardware, your result is unlikely to be the same as in a VM (which is pretty “sanitary” by comparison). Conversely if I tried to PXE boot a UEFI VM it would likely work.
The issues appears to be the combination of: FOG(really iPXE) -> UEFI boot on specific hardware -> ESXi
I am attempting ESXi 6.7 now to see if its the same issue. I will likely try another type/model of Dell server to see if that has the same issue.
Basically, I need to be able to UEFI PXE boot and install CentOS 7 and ESXi 6.5/6.7. I have CentOS working on the same hardware (Dell PowerEdge R620) but I get the PSOD when trying to install ESXi.
I hope this clarifies the situation.
-
@zer0cool OK, just to finish what I started, then we’ll focus on what I think you issue is.
I can pxe boot a Dell Optiplex 9020 in uefi mode into the esxi 6.5u1 installer and successfully install ESXi on the internal sata drive (not the typical install). ESXi 6.5 boots and is up and functional. So iPXE booting into uefi mode works perfectly.
Full disclosure, I had to switch to a 9020 because the 7010 is known for having flaky uefi firmware and its fails to boot the ipxe.efi boot loader in uefi mode.
Now to your issue. I might recommend that you use the DELL/EMC ESXi 6.5 installer to ensure you get all of the required Dell Server drivers. The PSOD may be related to a missing server driver and not related to pxe booting. But that is only a guess.
I have an old R410 I could test this process on, but that is a bit more effort than I have at the moment.
When you do, you will see this logo in vCenter.
-
@george1421 Hey Thanks for looking into it.
Hold off on anything more for now. ESXi 6.7 goes through the install without the PSOD on the same hardware.
I also should have mentioned these are both the Dell ESXi versions (6.5 U1 A10 and 6.7 A00)
I am seeing right now if 6.7 really works, or fails at some other point.
I am also xfering the 6.5 ISO to the server, and will mount/copy the files over from the server instead of on another machine and then ftp’ing them over in case I am somehow causing issues with the files.
Ill report my findings back. Also, I may try on another server to see if its just a quirk with the R620’s.
Thanks
-
@george1421 I can confirm ESXi 6.7 installs and boots without an issue.
I am going to proceed with mounting the ISO on the server for 6.5, copy its contents over into the proper place again, and see if it resolves anything.
If not, I am going to chalk it up to a compatibility issue between the R620, iPXE and ESXi 6.5 U1.
-
Well, I mounted the ISO for 6.5 on the server, copied everything over, recreated the menu and edited the boot.cfg and the same issue:
- The bootx64.efi and boot.cfg load as expected
- The VMWare ESXi installer loads files according to the boot.cfg
- The VMWare ESXi installer starts, the band/progress bar at the bottom gets past a couple stages and then PSOD on the “storage-stack” portion every time.
(all related to 6.5 U1 A10)
Tested 2x R620’s, one with all newest firmware, both have same issue. Tested on an FC630, works no issues.ISO burned to physical media runs as expected and works in both R620’s via UEFI DVD/CD boot.
I am going to try and alter the 6.5 install to run over ftp instead of http to see if it changes anything. I had done this prior I think, but want to be sure. I will pass everything via ftp (bootx64.efi, boot.cfg and all modules from boot.cfg).
If that doesnt work I will just give up on the R620’s getting PXE booted ESXi 6.5 U1 A10. As long as its 1 or 2 machines/models that arent critical I can live with that but if the crashes become the rule and not the exception thats going to cause issues for me.
Lastly, is there anyway to via UEFI do something similar to a memdisk/iso load? I know memdisk itself is BIOS only, is there a way to UEFI boot straight from ISO?
Thanks
EDIT: ftp crashes just the same, i give up unless I can UEFI boot the ISO
