Lenovo 300e ipxe boots ok but ipxe.efi can't resolve the system

mjaskowski

We just got 28 new Lenovo 300e WinBooks for our Math tutors that we’d like to clone with FOG with the following specs:

BIOS: 6HCN23WW
EC: 6HEC23WW
MTM: 81FY000BUS
CPU: Intel Celeron CPU N3450 @ 1.10GHz
Sys Memory: 4096 MB
eMMC Card: SanDisk iNAND 128GB
OS: Win10 Pro Education
Secure Boot Status: Disabled

No built-in NIC so I’m using a lenovo ThinkPad USB 3.0 Ethernet Adapter Model: PN:RTL8153

We can reach the Fog server registration page and start a Quick Registration but it never completes and neither does the Client System Information (Compatibility), the screen goes black and we get nothing after that. It will just sit that way for hours until I shut it down. Then I tried a manual registration which does get recognized and an image capture appears to start but also stalls and never proceeds.

I’ve even tried Acronis and Clonezilla already which load and then fail to find a hard drive. These units have 128gb eMMCs as primary drives.

Please advise. Thanks.

george1421

What version of FOG are you using?

So are you saying that iPXE menu displays and you can pick registration. From there you see iPXE transfer bzImage and init.xz but then the system just sits forever? Do I understand that right?

mjaskowski

Yes, right after choosing Quick Registration and Inventory I see bzImage… 100% ok and next init.xz… 100% ok the screen goes black and nothing more happens and it never appears in All Hosts.

mjaskowski

We are using FOG Version 1.5.6

george1421

@mjaskowski Ok the first step is to make sure the lenovo’s bios is up to date. This isn’t the first time we’ve seen this issue with lenovo systems. Sometimes a firmware update resolves the issue other times we’ve had to resort to booting FOS Linux from a usb flash drive. Either way we should be able to get you going with imaging.

mjaskowski

Okay. BIOS updated from 6HCN23WW to 6HCN24WW (current) but it made no difference. Next steps?

Sebastian Roth

@mjaskowski Increase the kernel log level (FOG Configuration -> FOG Settings -> FOG Boot Settings -> KERNEL LOGLEVEL) all the way up to 7 and boot the host machine to see if you get any further output from the kernel.

mjaskowski

I increased the kernel log level to 7 and updated FOG to 1.5.7 that in turn updated the kernel to 4.19.48 and received no further output or change. I even disabled the Intel Platform Trust Technology in the BIOS but still no difference.

Sebastian Roth

@mjaskowski Well then we’d need to build a special kernel image with EARLYPRINTK enabled to see if it’s actually able to load the kernel. Download the kernel here: https://fogproject.org/kernels/bzImage-earlyprintk and put into /var/www/html/fog/service/ipxe/ on your FOG server. Manually register the host’s MAC in the web UI, edit the settings and tell it to use bzImage-earlyprintk as Host Kernel. As well you need to set Host Kernel Arguments to earlyprintk=efi loglevel=7 (the later one just to make sure, probably not needed). Schedule a task for the host and boot it up to see what we get on screen.

mjaskowski

Done. I got the following:

Configuring (net0 M:A:C) … ok
Received DHCP answer on interface net0
tftp://I.P/default.ipxe… ok
http://I.P/fog/service/ipxe/boot.php… ok
bzImage-earlyprintk… ok
init.xz… ok
We made it to the Linux Kernel

But it hangs after that. No more feedback nor activity.

Sebastian Roth

@mjaskowski This is great news! We know it made it to load and execute the kernel. We will need to add more debugging output to see where exactly it locks up. We have done this several times with different hardware and I am fairly sure we will figure this one out as well. I won’t get to connect to my compile system until later on at night. So you will need to be patient or start learning how to compile your own kernel. It’s not that hard really.

git clone https://github.com/FOGProject/fos
cd fos
export KERNEL_VERSION=4.19.48
./build.sh -k -a x64
...

When this runs through without an issue you should see a message at the end telling you the kernel bzImage is ready.

Now what you want to do is edit .config file, search for “EARLY_PRINTK” and enable this line (remove # at the beginning and add =y at the end just as you see with other lines.

As well edit arch/x86/boot/compressed/eboot.c and search for the function called efi_main. Add print statements like efi_printk(sys_table, "Text output\n"); at various places in that function to find out where exactly it locks up.

mjaskowski

[root@MCS-FOG fos]# export KERNEL_VERSION=4.19.48
[root@MCS-FOG fos]# ./build.sh -k -a x64
Checking packages needed for building

Package meld missing!
Package ncurses-devel missing!
Package bison missing!
Package flex missing!
Package gcc-arm-linux-gnu missing!
Package gcc-aarch64-linux-gnu missing!
Package(s) missing, can’t build, exiting now.

mjaskowski

Okay. Installed all missing packages and tried again, at the end was prompted “We are ready to build. Would you like to edit the config file [y|n]?y”, which put me in .config - Linux/x86 4.19.48 Kernel Configuration in a blue bordered editor.
In this editor I only have options to do the following:
lqqqqqqqqqqqqqqqqqq Linux/x86 4.19.48 Kernel Configuration qqqqqqqqqqqqqqqqqqk
x Arrow keys navigate the menu. <Enter> selects submenus —> (or empty x
x submenus ----). Highlighted letters are hotkeys. Pressing <Y> includes, x
x <N> excludes, <M> modularizes features. Press <Esc><Esc> to exit, <?> x
x for Help, </> for Search. Legend: [*] built-in [ ] excluded <M> module x

I searched for “EARLY_PRINTK”
Search Results
x Symbol: EARLY_PRINTK [=n] x
x Type : bool x
x Prompt: Early printk x
x Location: x
x (1) -> Kernel hacking x
x Defined at arch/x86/Kconfig.debug:17 x
x x
x x
x Symbol: EARLY_PRINTK_DBGP [=n] x
x Type : bool x
x Prompt: Early printk via EHCI debug port x
x Location: x
x -> Kernel hacking x
x (2) -> Early printk (EARLY_PRINTK [=n]) x
x Defined at arch/x86/Kconfig.debug:30 x
x Depends on: EARLY_PRINTK [=n] && PCI [=y] x
x Selects: EARLY_PRINTK_USB [=n] x
x x
x x
x Symbol: EARLY_PRINTK_EFI [=n] x
x Type : bool x
x Prompt: Early printk via the EFI framebuffer x
x Location: x
x -> Kernel hacking x
x (3) -> Early printk (EARLY_PRINTK [=n]) x
x Defined at arch/x86/Kconfig.debug:43 x
x Depends on: EFI [=y] && EARLY_PRINTK [=n] x
x Selects: FONT_SUPPORT [=y] x
x x
x x
x Symbol: EARLY_PRINTK_USB [=n] x
x Type : bool x
x Defined at arch/x86/Kconfig.debug:6 x
x Selected by [n]: x
x - EARLY_PRINTK_DBGP [=n] && EARLY_PRINTK [=n] && PCI [=y] x
x - EARLY_PRINTK_USB_XDBC [=n] && EARLY_PRINTK [=n] && PCI [=y] x
x x
x x
x Symbol: EARLY_PRINTK_USB_XDBC [=n] x
x Type : bool x
x Prompt: Early printk via the xHCI debug port x
x Location: x
x -> Kernel hacking x
x (4) -> Early printk (EARLY_PRINTK [=n]) x
x Defined at arch/x86/Kconfig.debug:53 x
x Depends on: EARLY_PRINTK [=n] && PCI [=y] x
x Selects: EARLY_PRINTK_USB [=n] x

So not sure how to proceed.

mjaskowski

In this editor I navigated to Kernel hacking and set [*] Early printk and saved. When searched again I find:
x Symbol: EARLY_PRINTK [=y] x
x Type : bool x
x Prompt: Early printk x
x Location: x
x (1) -> Kernel hacking x
x Defined at arch/x86/Kconfig.debug:17 x

All the rest still [=n] I hope that’s what you mean and the only one you mean. I’ll try to edit eboot.c now.

mjaskowski

In eboot.c using vi I don’t see a function called efi_main
/* -----------------------------------------------------------------------
*

Copyright 2011 Intel Corporation; author Matt Fleming
This file is part of the Linux kernel, and is made available under
the terms of the GNU General Public License version 2.
----------------------------------------------------------------------- */

#include <linux/efi.h>
#include <linux/pci.h>

#include <asm/efi.h>
#include <asm/e820/types.h>
#include <asm/setup.h>
#include <asm/desc.h>

#include “…/string.h”
#include “eboot.h”

static efi_system_table_t *sys_table;

static struct efi_config *efi_early;

__pure const struct efi_config *__efi_early(void)
{
return efi_early;
}

#define BOOT_SERVICES(bits)
static void setup_boot_services##bits(struct efi_config *c)
{
efi_system_table_##bits##_t *table;

table = (typeof(table))sys_table;

c->runtime_services = table->runtime;
c->boot_services = table->boottime; \

mjaskowski

Okay. I believe I misunderstood when to edit the files, you mean after a successful build. In the build I hit one more error: Missing elfutils-libelf-devel so I installed that and started again and it’s building for a while now.

Sebastian Roth

@mjaskowski Sorry, didn’t get to answer until now. Great to see you are making good progress on this!!

Not sure exactly why but turns out I missed to to EARLY_PRINTK_EFI in the kernel I provided earlier and it still seemed to work for you. So not sure if this is needed or not. In case you don’t get the test output as expected you might want to add the EARLY_PRINTK_EFI as well and try again.

In eboot.c using vi I don’t see a function called efi_main

Do you know how to search in VI? Make sure you are not in edit mode (hit ESC a couple of times) and type /efi_main then ENTER. It should jump to the first occurrence. If you want to step to the next hit n or shift+n to jump back. If this doesn’t work, just use page down to go to line 740. Should look like this: https://github.com/torvalds/linux/blob/master/arch/x86/boot/compressed/eboot.c#L740

Okay. I believe I misunderstood when to edit the files, you mean after a successful build.

Nope, before building. But it’s no big deal! One important thing I should mention is that build.sh is preparing a lot of things for you to make the build work. But as soon as you have called it once you can also go the manual route. This is quite often better.

So to change the config in GUI mode do:

cd fos/kernelsourcex64/
make menuconfig

And to build the kernel just do:

cd fos/kernelsourcex64/
make bzImage

You can repeat the make bzImage over and over every time you make a change to the eboot.c code to get a new bzImage binary that has the code changes in it.

Sebastian Roth

@mjaskowski And when the kernel build fine you just need to copy it over to your FOG server in /var/www/html/fog/service/ipxe/. And I’d still name it bzImage-earlyprintk so you don’t overwrite the original kernel binary.

mjaskowski

the last build failed with MK_FW firmware/bnx2x/bnx2x-e1h-7.13.1.0.fw.gen.S
make[1]: *** No rule to make target linux-firmware/bnx2x/bnx2x-e1h-7.13.1.0.fw', needed by firmware/bnx2x/bnx2x-e1h-7.13.1.0.fw.gen.o'. Stop.
make: *** [firmware] Error 2
Since I’m starting it over I’ll edit those files prebuild but what about this error, how do I prevent a repeat?

george1421

@mjaskowski said in Lenovo 300e ipxe boots ok but ipxe.efi can't resolve the system:

make[1]: *** No rule to make target linux-firmware/bnx2x/bnx2x-e1h-7.13.1.0.fw’, needed by firmware/bnx2x/bnx2x-e1h-7.13.1.0.fw.gen.o’. Stop.

I’ve done my share of linux kernel building (watch that statement come back and bit me in the backside). What this is saying is that you don’'t have the linux kernel firmware downloaded and saved into the root of the directory where the .config file is.

git this: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git

Lenovo 300e ipxe boots ok but ipxe.efi can't resolve the system

We made it to the Linux Kernel

162

12.0k

17.3k

155.2k