FOG menu don't boot Windows - Chainloading failed

[Sebastian Roth]

@thomasdec Thanks for all the effort you put into debugging this! The pictures are perfect! Good that you found out about the dhcp command. I forgot to tell you. The yellow NETDEV messages appear because packets from your network arrive that iPXE is ignoring (as described in one of my earlier posts). So that’s ok.

Interestingly enough it seams to boil down to “bzImage is not an EFI”. In a chat session we downloaded and checksum tested the bzImage by hand again but run into the same error!! To me this feels like the bzImage is being corrupted when transfered from the FOG server to the client. We have talked about adding the trusted check to iPXE but we never did because pretty much all those issues disappeared right after Tom added the checksums to the installer not long ago! I am really wondering if I am going blind here or if the bzImage is actually being corrupted on the wire? Any ideas?

[george1421]

@Sebastian-Roth I still wonder if there IS an issue with iPXE transferring the FOS kernel to the client. You are right we did testing not to long ago where the bzImage would not load on a vmclient in efi mode. Then Tom did something (not related to checksum) and it started working in my test environment. I think that was FOS kernel 4.5.0 (or about there).

For this Lenovo Yoga, would having the OP build a grub boot flash drive give us any useful intelligence? I think it would give us an idea if the bzImage is at fault. The issue is the handoff between grub and ipxe to the bzImage is a bit different so it wouldn’t be a true like for like test.

[Sebastian Roth]

@george1421 I thought about flash drive testing as well but as you already said this is quite different in how the kernel is loaded in UEFI mode. The only test that makes any sense from my point of view is loading the kernel straight from a usb key. None of the other methods would actually enter the kernel trough the EFI_STUB. I really hope that we haven’t found another UEFI firmare bug (like with the HP x2 210 - which is still not solved although I have a direct contact to those firmware guys now!!)

@thomasdec I know you are in the weekend already but could you please try this when you get back next week? Get a usb key, empty, format with FAT/VFAT/FAT32 and put the bzImage from your FOG server on the usb key. The path and filename need to be exactly like this: EFI\BOOT\BOOTX64.EFI (windows) - EFI/BOOT/BOOTX64.EFI (if your prepare the usb key on a linux machine, both should work!). Then boot your Yoga from this usb key in (UEFI mode). Either you will see an error from the firmware or plain reboot or hang. Then I guess we have a firmware bug. Otherwise you will see some kernel boot messages and then a kernel panic - which is totally fine because booting the bzImage on its own does not work. But at least we see that the kernel is loading…

[george1421]

@Sebastian-Roth Is there any value in seeing if the OP an boot from a live linux image in efi mode, like the ubuntu desktop live?

(Just thinking out loud here)
Now that I think about it, it would also use grub to boot that image, so that wouldn’t be a good test. And having us efi boot ipxe from the flash drive then pull the bzImage from the server, would give us the same results as pxe booting. What would be great is if we could tell the ipxe kernel to load the bzImage from the local flash drive instead of the network.

[thomasdec]

@Sebastian-Roth I have a kernel panic on USB boot.

[Sebastian Roth]

@thomasdec Here is a signature file bzImage.sig for the current kernel image ( @Tom-Elliott I hope you don’t intend to update the official kernel images right now - would break my signature file…).

As well you need a “special” ipxe.efi binary (CA cert included).

[thomasdec]

@Sebastian-Roth Result of your modify kernel :

After this screen, windows boot normally

[Sebastian Roth]

@thomasdec Once again… try this ipxe.efi

[thomasdec]

@Sebastian-Roth Result :

[Sebastian Roth]

@thomasdec This is another very weird thing here LoadImage (EFI) function is not able to lead the bzImage although it seams to be intact (file size and signature checked)! So we need to dig into the EFI calls. Please try this ipxe.efi (compiled with DEBUG=efi_wrap)

[Sebastian Roth]

@thomasdec Have you ever tried booting other UEFI machines?? Please do so! We wanna make sure if this is an issue with FOG/network/setup or if it is a client/Yoga issue!?

[Sebastian Roth]

@thomasdec As well please run the following ipxe.efi (compiled with DEBUG=efi_image) to see what the exact error from LoadImage call is.

[thomasdec]

@Sebastian-Roth I tested with other Yoga 11e and ThinkPad E550 in UEFI mode, I have the same result.
Picture from test with DEBUG=efi_image :

In Legacy mode I can image my windows, so for now I will use this solution to develop my sysprep.

[Sebastian Roth]

@thomasdec Getting there step by step (I hope…). Please try this ipxe.efi where I added output of the “real” UEFI return code - iPXE usually only prints their own error codes which don’t give us more information. Mind you the output will be very similar to the last one but it should say:

EFIIMAGE 0x... could not load, efirc=??: Error...

I tested with other Yoga 11e and ThinkPad E550 in UEFI mode, I have the same result.

This - and the fact that the kernel booted from USB key - make me think that this is NOT an iPXE/UEFI issue but just some weird problem in your setup - hope we will find it soon!!!

[thomasdec]

@Sebastian-Roth I haveefirc=1 on all EFIIMAGE message.

[george1421]

@thomasdec Hey, I’d just like to thank you for hanging in here with us. We really need to get to the root of the issue with these “broken” efi firmware. Its not really broken, there is just something that iPXE is not handing off correctly under all efi firmware. Most efi firmware works correctly, there are a few systems that it is not. To get to the root of the problem we need people like you who are willing to hang in there with us to see if we can pinpoint the conflict.

Again I’d like to say thank you for helping with the try this, change that process. In the end this info will help the iPXE project as well as the FOG project to deliver a better product.

[thomasdec]

I found why the .efi file doesn’t work !
The problem come from an boot option.
I disable Security Chip in Security tab in Lenovo BIOS and now it’s work.
I can access FOG menu and image my computer.
Sorry for not finding it sooner !

@Sebastian-Roth @george1421 thanks again so much for your help !

[george1421]

Marking this Solved.

Great job finding that setting. I’m sure it will help someone else. I think there is another thread that will end up at this conclusion too.

[Sebastian Roth]

@thomasdec Thank god you found this. Looking at the PCAP file we captured I was pretty much convinced that it’s something weird on the file transfer as I saw retransmissions and as well missing packets… (possibly just a screwed packet dump?!?).

[george1421]

@Sebastian-Roth Is there something we can learn/document from this issue?

i.e. if you do X and you get results Y and Z and you are running hardware X, please check the security chip setting? There has to be a learning point in this thread.