Hardware Inventory task hangs

[chunter2]

I tried running the Hardware Inventory task and it seems to hang. I’m running fog 1.5.6 and was able to schedule the task on a reboot. The host rebooted and started what I think is the beginning of the Hardware Inventory task. I have the following on the screen.

iPXE 1.0.0+ (9907f) – …
Features: …
Configuring (net0 MACaddress)…ok
Received DHCP answer on interface net0
tftp://ipOfServer/default.ipxe… ok
http://ipOfServer/fog/service/ipxe/boot.php… ok
bzImage_412… ok
init.xz… ok
flashing cursor

This is on a Dell OptiPlex 3040 if that matters.

Any ideas?

Thanks

[Sebastian Roth]

@chunter2 said:

This is on a Dell OptiPlex 3040 if that matters.

I am fairly sure this is specific to this hardware. Have you tried other machines as well?

We can start digging into this using a special kernel image with EARLYPRINTK enabled to see if it’s actually able to load and start the kernel. Download the kernel here: https://fogproject.org/kernels/bzImage-earlyprintk and put into /var/www/html/fog/service/ipxe/ on your FOG server. Manually register the host’s MAC in the web UI, edit the settings and tell it to use bzImage-earlyprintk as Host Kernel. As well you need to set Host Kernel Arguments to earlyprintk=efi loglevel=7 (the later one just to make sure, probably not needed). Then try booting into the inventory task.

[chunter2]

Now I’m confused. That worked perfectly. It booted into the kernel you had me download, ran the hardware inventory and rebooted. I then went to the inventory tab in the web UI and it was updated. I had updated the bios on the OptiPlex 3040 so was looking for the updated bios version number and it’s there.

I did notice yesterday after I posted that disk surface test and debug tasks also hung. I just tried a disk surface test now with the downloaded kernel and it’s working too. Not sure what to think or what information would be helpful.

Thanks

[george1421]

@chunter2 OK I’m a bit confused here. Why are you using bzImage_412 (it seems like I had a hand in that) with FOG 1.5.6. If I had a hand in the matter, I would have only recommended 4.12.x to people who were on FOG 1.4.4 and didn’t want to migrate. This was to give the user access to the newish intel network drivers.

For that new(ish) hardware Dell 3040, you should be on 4.19.X series kernels. Now using a current level kernel may not address your issue, its just something inconsistent I’m seeing in what you’ve told us.

[chunter2]

@george1421 I’m not actually sure why it’s using bzImage_412. I upgraded from fog 1.2.0. Maybe the bzImage didn’t get upgraded? I did go to FOG Configuration and tried the kernel update. I chose 4.19.64 TomElliott 64 and it seemed to say it updated in the web UI. The bzImage in my /var/www/html/fog/service/ipxe doesn’t seem to be newer than my upgrade date. Should it be?

[george1421]

@chunter2 The call for bzImage412 would be listed globally in the FOG Configuration -> FOG Settings page or in the kernel parameter of the host definition for this specific host.

As for the updated kernel if you go to /var/www/html/fog/service/ipxe and key in file bzImage it will tell you the kernel version.

[chunter2]

@george1421 In the settings under TFTP Server the TFTP PXE KERNEL was set to bzImage_412. I’m assuming it should be bzImage? What’s strange is there’s no bzImage_412 file in the /var/www/html/fog/service/ipxe folder.

I ran file bzImage and got the following.

bzImage: Linux kernel x86 boot executable bzImage, version 4.19.36 (jenkins-agent@Tollana) #1 SMP Sun Apr 28 18:10:07 CDT 2019, RO-rootFS, swap_dev 0x7, Normal VGA

Not sure why it’s not showing 4.19.48 since I thought I updated it.

[george1421]

@chunter2 Hey sorry for the delay, Yes that setting should be bzImage (watch the case) for the 64 bit kernel and bzImage32 for the 32 bit kernel.

Kernel 4.19.36 is good enough for the 3040. Its not clear why the kernel update didn’t replace that kernel for you. FWIW you can manually download the kernels from here too: https://fogproject.org/kernels/ You will just need to rename the downloaded files to bzImage and bzImage32 respectively.

if bzImage412 is missing I would expect you to have problems right away since that IS the FOS Linux kernel. Without that the target system won’t boot.

[chunter2]

@george1421 Turns out the TFTP PXE KERNEL DIR variable was set to /var/www/fog/service/ipxe instead of /var/www/html/fog/service/ipxe/ which explains why the kernel wasn’t updating. Now it is. And I’ve got 4.19.64 now. I’m not sure why it was set wrong in the first place. And I agree, I don’t know why hosts were still booting. I’d get the fogproject splash page with a 3 second countdown on booting the first hard drive and then they would boot normally. A memory test seemed to work from the splash page but nothing else that I tried.

[george1421]

@chunter2 Interesting because the directory /var/www/fog should be linked to /var/www/html/fog The issue was ubuntu had one path and centos/rhel had a different path. So the developers created the link directory to satisfy both camps.

So now does things work or not? For this 3040 are can you run the FOG compatibility test from the iPXE menu?

[chunter2]

@george1421 Right now I’ve got all the web files in /var/www/html/fog. Inside that directory there’s a symlink called fog pointing to /var/www/html/fog (same thing). Not sure why. I’m on Ubuntu 18.04. How should it be set up so upgrades work properly?

I’ll have to test tonight to make sure the hosts are booting correctly and that the compatibility test works but I’m pretty sure they will now.

[george1421]

@chunter2 Rerunning the FOG installer “should” have fixed any sins from the past (with the exception of custom settings in the fog configurations like your kernel name)

[chunter2]

@george1421 All my hosts seem to be working fine now.

Should I just remove the useless symlink?

Thanks for the help.

[george1421]

@chunter2 Leaving it won’t hurt anything. One might think, if its working now its good enough.