Black screen and blinking cursor after vesamenu

Luc Novales

@Sebastian-Roth said in Black screen and blinking cursor after vesamenu:

@Luc-Novales Can you tell us more about this old hardware. What make and model is this?

This model is an IEI AFL-07A-LX : a panel PC used in an operational equipment.

System is detected as :

dmidecode: System Information
dmidecode: 	Manufacturer: ICP / iEi
dmidecode: 	Product Name: H605
dmidecode: 	Version: V1.00
...
dmidecode: Base Board Information
dmidecode: 	Manufacturer: ICP / iEi
dmidecode: 	Product Name: E089
dmidecode: 	Version: V1.00

This model is AMD Geode LX 800 500 MHz Processor
It is detected under linux as :

/proc/cpuinfo: processor	: 0
/proc/cpuinfo: vendor_id	: AuthenticAMD
/proc/cpuinfo: cpu family	: 5
/proc/cpuinfo: model		: 10
/proc/cpuinfo: model name	: Geode(TM) Integrated Processor by AMD PCS
/proc/cpuinfo: stepping	: 2
/proc/cpuinfo: cpu MHz		: 498.042
/proc/cpuinfo: cache size	: 128 KB
/proc/cpuinfo: physical id	: 0
/proc/cpuinfo: siblings	: 1
/proc/cpuinfo: core id		: 0
/proc/cpuinfo: cpu cores	: 1
/proc/cpuinfo: apicid		: 0
/proc/cpuinfo: initial apicid	: 0
/proc/cpuinfo: fdiv_bug	: no
/proc/cpuinfo: f00f_bug	: no
/proc/cpuinfo: coma_bug	: no
/proc/cpuinfo: fpu		: yes
/proc/cpuinfo: fpu_exception	: yes
/proc/cpuinfo: cpuid level	: 1
/proc/cpuinfo: wp		: yes
/proc/cpuinfo: flags		: fpu de pse tsc msr cx8 sep pge cmov clflush mmx mmxext 3dnowext 3dnow cpuid 3dnowprefetch vmmcall
/proc/cpuinfo: bugs		: sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass
/proc/cpuinfo: bogomips	: 996.08
/proc/cpuinfo: clflush size	: 32
/proc/cpuinfo: cache_alignment	: 32
/proc/cpuinfo: address sizes	: 32 bits physical, 32 bits virtual
/proc/cpuinfo: power management:
/proc/cpuinfo:

and dmidecode defines CPU as 80486 family

dmidecode: Processor Information
dmidecode: 	Socket Designation: Geode LX 
dmidecode: 	Type: Central Processor
dmidecode: 	Family: 80486
dmidecode: 	Manufacturer: AMD
dmidecode: 	ID: A2 05 00 00 3D A9 88 00
dmidecode: 	Signature: Type 0, Family 5, Model 10, Stepping 2
dmidecode: 	Version: Geode LX
dmidecode: 	Voltage: 1.2 V
dmidecode: 	External Clock: 64 MHz
dmidecode: 	Max Speed: 500 MHz
dmidecode: 	Current Speed: 500 MHz
dmidecode: 	Status: Populated, Enabled
dmidecode: 	Upgrade: ZIF Socket
dmidecode: 	L1 Cache Handle: 0x0004
dmidecode: 	L2 Cache Handle: 0x0005
dmidecode: 	L3 Cache Handle: No L3 Cache

BIOS information is :

dmidecode: BIOS Information
dmidecode: 	Vendor: Phoenix Technologies, LTD
dmidecode: 	Version: 6.00 PG
dmidecode: 	Release Date: 05/18/2010
dmidecode: 	Address: 0xE0000
dmidecode: 	Runtime Size: 128 kB
dmidecode: 	ROM Size: 512 kB
dmidecode: 	Characteristics:
dmidecode: 		ISA is supported
dmidecode: 		PCI is supported
dmidecode: 		PNP is supported
dmidecode: 		APM is supported
dmidecode: 		BIOS is upgradeable
dmidecode: 		BIOS shadowing is allowed
dmidecode: 		Boot from CD is supported
dmidecode: 		Selectable boot is supported
dmidecode: 		BIOS ROM is socketed
dmidecode: 		EDD is supported
dmidecode: 		5.25"/360 kB floppy services are supported (int 13h)
dmidecode: 		5.25"/1.2 MB floppy services are supported (int 13h)
dmidecode: 		3.5"/720 kB floppy services are supported (int 13h)
dmidecode: 		3.5"/2.88 MB floppy services are supported (int 13h)
dmidecode: 		Print screen service is supported (int 5h)
dmidecode: 		8042 keyboard services are supported (int 9h)
dmidecode: 		Serial services are supported (int 14h)
dmidecode: 		Printer services are supported (int 17h)
dmidecode: 		CGA/mono video services are supported (int 10h)
dmidecode: 		ACPI is supported
dmidecode: 		LS-120 boot is supported

Others information as memory… in attached hardware-summary.txt.

I posted in the iPXE forum to see if they can help us debugging this. So far I have only done debugging in the higher level code and don’t know where to start here in the very early assembly code: http://forum.ipxe.org/showthread.php?tid=18855

Thanks,
Luc.

Luc Novales

Hi,
I can add some information.

@Luc-Novales said in Black screen and blinking cursor after vesamenu:

Is there some way to debug without compiling IPXE ?

I made iPXE USB-KEY using this howto, it doesn’t boot and stop with blinking cursor after “Verifying Pool Data…”, nothing done after, but it is due to this platform because this key boots on an other PC.
Adding some debug options did not change anything (no messages), perhaps I selected bad options ?

Even I build ipxelinux.0 as defined here, result is the same as using undionly.kkpxe file from FOG install.

Luc.

Sebastian Roth

@Luc-Novales Unfortunately there is still no answer in the iPXE forum. Not sure what else we can try.

Are you good with C and assembly code?

george1421

I just scanned this thread and all of the pictures I see for booting are syslinux not iPXE. The processor is 32 bits in size since its a 486 equiv, so that might make a difference. The ipxe.krn is (should be) a 16 bit application that should run on a 486 class machine.

So if iPXE is being a pain why not boot FOS directly from a usb stick. You will have to use the bzImage32 and init32.xz files instead of the 64 bit ones, but lets see if FOS Linux will boot on this system.

Luc Novales

@Sebastian-Roth said in Black screen and blinking cursor after vesamenu:

@Luc-Novales Unfortunately there is still no answer in the iPXE forum. Not sure what else we can try.

Are you good with C and assembly code?

Not really :-/.

Luc Novales

Hi George,

Thanks for your help,

@george1421 said in Black screen and blinking cursor after vesamenu:

I just scanned this thread and all of the pictures I see for booting are syslinux not iPXE. The processor is 32 bits in size since its a 486 equiv, so that might make a difference. The ipxe.krn is (should be) a 16 bit application that should run on a 486 class machine.

It doesn’t start

So if iPXE is being a pain why not boot FOS directly from a usb stick. You will have to use the bzImage32 and init32.xz files instead of the 64 bit ones, but lets see if FOS Linux will boot on this system.

FOS Linux ???

iPXE doesn’t boot, PXE yes. I tried to make iPXE usb stick just for test and it doesn’t boot on this panel PC.
Usb stick is not a definitive solution to deploy these machines in a training room between two sessions. I have to integrate these equipment in our environment as the others PC used to deploy images.
I can boot linux kernel from PXE. Debug iPXE should be a better solution to download and chain “http://<fog server>/fog/service/ipxe/boot.php”, but I don’t know howto…

Luc.

george1421

@Luc-Novales Ok lets take a step back because I haven’t seen a picture yet from iPXE. Chain booting from syslinux to iPXE won’t work because you say that iPXE is hanging. Using syslinux to call boot.php won’t work since boot.php will make a menu for iPXE and not Syslinux. I suspect that pxelinux.0 (syslinux) will not like an iPXE style menu.

Lets first set dhcp options 67 to ipxe.kkpxe (this exact boot file) and pxe boot the target system. Using a mobile phone, take a screen shot of the error produced by ipxe (I know you have done this many times, but I want to see the error as well as the context of the error).

FOS Linux: This is the capture engine that runs on the target computer to capture and deploy images on the target hardware. Normally it is sent to the target hardware using iPXE via the menu produced by the boot.php page. FOS Linux is constructed out of 2 parts bzImage32 is the Linux 32 bit kernel and init_32.xz is the 32 bit virtual hard drive. Its iPXE’s job to transfer them to the target computer.

Today we have a usb flash drive that can be used to transfer bzImage32 and init_32.xz to the target computer using a usb flash drive and a grub boot loader.

Now with that said, we should be able to duplicate that grub boot menu using syslinux (pxelinux.0) Its not an ideal solution but if you have no choice, sometimes you have to do what you have to do to make things work. So if we can’t get things working with iPXE, then we can attempt to use syslinux to at least get FOS Linux to the target computer.

I know we are focusing on the CPU in this thread, but something I have to ask How much RAM does this device have?

Only for reference so I don’t forget and lose it.

default menu.c32
prompt 0
timeout 300
ONTIMEOUT local

MENU TITLE FOG PXE Menu

LABEL 1. FOG Image Deploy/Capture
        MENU LABEL 1. FOG Image Deploy/Capture
        KERNEL bzImage32
	APPEND loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4

LABEL 2. Perform Full Host Registration and Inventory
        MENU LABEL 2. Perform Full Host Registration and Inventory
        KERNEL bzImage32
        APPEND loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=manreg

LABEL 3. Quick Registration and Inventory
        MENU LABEL 3. Quick Registration and Inventory
        KERNEL bzImage32
        APPEND loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=autoreg

LABEL 4. Client System Information (Compatibility)
        MENU LABEL 4. Client System Information (Compatibility)
        KERNEL bzImage32
        APPEND loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=sysinfo

LABEL 5. FOG Debug Kernel
        MENU LABEL 5. FOG Debug Kernel
        KERNEL bzImage32
        APPEND loglevel=7 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= boottype=usb consoleblank=0 rootfstype=ext4 isdebug=yes

Luc Novales

@george1421 said in Black screen and blinking cursor after vesamenu:

@Luc-Novales Ok lets take a step back because I haven’t seen a picture yet from iPXE. Chain booting from syslinux to iPXE won’t work because you say that iPXE is hanging. Using syslinux to call boot.php won’t work since boot.php will make a menu for iPXE and not Syslinux. I suspect that pxelinux.0 (syslinux) will not like an iPXE style menu.

Lets first set dhcp options 67 to ipxe.kkpxe (this exact boot file) and pxe boot the target system. Using a mobile phone, take a screen shot of the error produced by ipxe (I know you have done this many times, but I want to see the error as well as the context of the error).

Here the screenshot resulting and tcpdump attached.
Boot_ipxe.kkpxe.jpg

And PCAP file is attached here
ipxe.kkpxe_2018-08-09.pcap

FOS Linux: This is the capture engine that runs on the target computer to capture and deploy images on the target hardware. Normally it is sent to the target hardware using iPXE via the menu produced by the boot.php page. FOS Linux is constructed out of 2 parts bzImage32 is the Linux 32 bit kernel and init_32.xz is the 32 bit virtual hard drive. Its iPXE’s job to transfer them to the target computer.

Ok.

Today we have a usb flash drive that can be used to transfer bzImage32 and init_32.xz to the target computer using a usb flash drive and a grub boot loader.

Is there a link to download or a script to make it from git clone ?
I didn’t see it.

Now with that said, we should be able to duplicate that grub boot menu using syslinux (pxelinux.0) Its not an ideal solution but if you have no choice, sometimes you have to do what you have to do to make things work. So if we can’t get things working with iPXE, then we can attempt to use syslinux to at least get FOS Linux to the target computer.

Good way for me. If I can deploy with syslinux before getting better solution with iPXE, I will. In a second time I can have help in C code

I know we are focusing on the CPU in this thread, but something I have to ask How much RAM does this device have?

1024MB of RAM.

/proc/meminfo: MemTotal:         994192 kB
/proc/meminfo: MemFree:          797440 kB
/proc/meminfo: MemAvailable:     738332 kB

from hardware-summary.txt

Only for reference so I don’t forget and lose it.

Ok.

Thx,
Luc.

Luc Novales

@george1421 said in Black screen and blinking cursor after vesamenu:

…

Only for reference so I don’t forget and lose it.

default menu.c32
prompt 0
timeout 300
ONTIMEOUT local

MENU TITLE FOG PXE Menu

LABEL 1. FOG Image Deploy/Capture
        MENU LABEL 1. FOG Image Deploy/Capture
        KERNEL bzImage32
	APPEND loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4

LABEL 2. Perform Full Host Registration and Inventory
        MENU LABEL 2. Perform Full Host Registration and Inventory
        KERNEL bzImage32
        APPEND loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=manreg

LABEL 3. Quick Registration and Inventory
        MENU LABEL 3. Quick Registration and Inventory
        KERNEL bzImage32
        APPEND loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=autoreg

LABEL 4. Client System Information (Compatibility)
        MENU LABEL 4. Client System Information (Compatibility)
        KERNEL bzImage32
        APPEND loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=sysinfo

LABEL 5. FOG Debug Kernel
        MENU LABEL 5. FOG Debug Kernel
        KERNEL bzImage32
        APPEND loglevel=7 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= boottype=usb consoleblank=0 rootfstype=ext4 isdebug=yes

Only for fun, I tried it, result is here

george1421

@Luc-Novales The syslinux screen tells is a bit more about what is going on than the previous pictures. Its possible that the minimum CPU level on both bzImage32 and the iPXE kernels are set to high for that older processor. We could try to build a new bzImage32 as a test to see.

george1421

Just as a follow up on this. The bzImage32 is configured to require a 686 as the minimum processor: https://github.com/FOGProject/fos/blob/master/configs/kernelx86.config#L251

I’ve reconfigured my build environment and changed the minimum processor to 486 and I’m rebuilding the bzImage32 file for i486. Understand two things. 1. This is not an official FOG Project kernel since its not coming from one of the Developers. 2. I have no idea if it will work because I don’t have a 486 machine to test it on.

When its done compiling I’ll post a link so you can download it and try it.

Also on my syslinux configuration, Its been almost 8 years since I’ve worked with syslinux so I guessed at most of the entries. You WILL need to adjust it to replace the entire variable $myfogip with the IP address of your fog server to make it work correctly. It will take about 20 minutes to finish the new kernel build.

george1421

Here is the link to the 486 version of the FOS Linux kernel: https://drive.google.com/open?id=1xSR5MIPoOTFDQNFpHs7flRUnjHqnJS-W

I’m going to suspect that you will have issues with the inits (virtual hard drive), but lets take one step at a time. Since I’ve given this a new name “bzImage486” you will need to update the syslinux menu too.

[edit] I just looked at the buildroot package and the default arch is set for i386 so the inits should work with this 486 kernel.

Sebastian Roth

@Luc-Novales We have an answer in the iPXE forum. I just compiles a debug enabled binary. Find that here: https://fogproject.org/ipxe/01_undionly.kpxe

Copy into /tfttpboot on your FOG server and give it a try. Take a picture if you get more messages on screen.

george1421

@Sebastian-Roth Is there a minimum processor level within the iPXE binaries, akin to where the kernel has minimum level supported processor?

Sebastian Roth

@george1421 said in Black screen and blinking cursor after vesamenu:

Is there a minimum processor level within the iPXE binaries, akin to where the kernel has minimum level supported processor?

Not sure about that. I don’t think so as iPXE doesn’t use modern CPU features much I guess and would therefore run on plain old 386 I suppose. But not exactly sure.

Sebastian Roth

@Luc-Novales Still waiting for you to test 01_undionly.kpxe! Please let us know.

Luc Novales

Hi,
I have to report some actions to explain where I am now.

Using IPXE

After a lot of tries 2 weeks ago, yesterday, I decide to compile IPXE “from scratch”, using a clean clone of ipxe git repository.

Applying “486 patch”

After removing definition TIVOLI_VMM_WORKAROUND in configuration file general.h as defined in the forum of ipxe “486er hangs on boot with iPXE floppy”, I obtain an USB boot media (ipxe.usb) to launch ipxe command and test FOG menu.

Compiling FOG IPXE

Compiling IPXE using IPXE part of the WIKI and applying “486 patch” (above) allow to obtain an undionly.kpxe usable with this panel PC.

Loading FOG menu

Configuring DHCP server to load new undionly.kpxe doesn’t load menu but allow to chain it manually.
Select “CLient System Information (Compatibiliy)” load effectively bzImage32 and stop with the message “This kernel requires an i686 CPU but only detected an i586 CPU”.

Using adapted linux kernel to i586

Trying to compile custom kernel, I opened another thread and George send me a bzImage file adapted to i486 (see FAQ page in WIKI should be updated).

Boot from FOG menu

Chain manually boot.php with MAC address parameter boot and the result is here

Fog menu for this host is here

TODO

Determine why undionly.kpxe doesn’t load menu
Determine why kernel loop on RCU grace-period stack dump.

Sebastian Roth

@Luc-Novales Don’t want to sound rude but how many of those panel PCs do you have? If it’s just a handful of them you might be way faster installing/cloning them manually using other means of deployment instead of trying to get FOS Linux to rnu on this kind of mul-functioning hardware.

Search the forums for “rcu_sched detected stalls on CPUs/tasks”…

Luc Novales

@Sebastian-Roth said in Black screen and blinking cursor after vesamenu:

@Luc-Novales Don’t want to sound rude but how many of those panel PCs do you have? If it’s just a handful of them you might be way faster installing/cloning them manually using other means of deployment instead of trying to get FOS Linux to rnu on this kind of mul-functioning hardware.

Today It’s the only reason to maintain an old server this an old version of clonezilla. All other stuff of our equipment in our learning rooms are managed on newer server with FOG. It should be better to integrate these panel PC too

Search the forums for “rcu_sched detected stalls on CPUs/tasks”…
I will…

In your signature :

…way more we need people to test things, update documentation…

Solving this problem, learning to compile new kernel… is perhaps the only way I have to update documentation and contribute to FOG

george1421

Ok update on this thread so we can put it to bed.

I built 2 custom FOS Linux kernels the fist was targeted to a 486 and the second was targeted to the Geode processor. Bot would throw the issue with an unstable clocksouce. We could only see this error when we change the linux kernel logging level to 7. After some google fu and attempting a few kernel parameters that dealt with the clocksouce I found a references to apci=off. That was the actual parameter that solved the kernel boot errors. The OP was then able to run a deployment in debug mode to completion.

So while the actual problem isn’t in line with the thread subject. The issues were this.

The standard FOS Linux kernel has a minimum supported level of processor of i586. I had to build a custom kernel with a minimum processor level of i486 to get the kernel to boot.
Because of the hardware we had to specifically turn off acpi in the kernel using the kernel parameter of apci=off

Black screen and blinking cursor after vesamenu

Using IPXE

Applying “486 patch”

Compiling FOG IPXE

Loading FOG menu

Using adapted linux kernel to i586

Boot from FOG menu

TODO

263

11.6k

17.1k

154.6k