kernel panic when imaging

[tlehrian]

I’m trying to image a computer that we fixed (replaced several components), but halfway through the imaging process I get a kernel panic, like the following:

Closeup of text:

I’m not sure whether the cause is hardware or on the fog server side.

Imaging HP Z2 G4
Fog version 1.5.10
bzImage and bzImage32 are both on version 6.1.63 (I originally tried the stock 5.15.93 but upgraded in case that was the issue).

Any ideas?

[george1421]

@tlehrian It is surely a hardware issue. The error message is not detailed enough to point at one thing unfortunately.

So lets start with the basics:
What I can glean from the error message is that this computer is using firmware version 1.01. Firmware 01.08.08 Rev.A was released on 21-Nov-23. I would start by updating the firmware.

If that doesn’t solve it then, swap memory with the working computer.

If that doesn’t solve it then, remove all non-essential hardware cards.

If that doesn’t solve it then download the 5.15.x version of the linux kernel for FOG. Edit: I see now that you were at 5.15 and then upgraded. So this isn’t the issue.

[tlehrian]

I’m in the process of trying to deploy the same image on another computer with known good hardware to try to rule out any issues with the image itself.

[tlehrian]

A successful deploy of this image on another computer indicates that it is not an issue with the image itself or with our server.

Any ideas what the cryptic messaging is saying? Which hardware caused this?

Thanks
Tim

[george1421]

@tlehrian It is surely a hardware issue. The error message is not detailed enough to point at one thing unfortunately.

So lets start with the basics:
What I can glean from the error message is that this computer is using firmware version 1.01. Firmware 01.08.08 Rev.A was released on 21-Nov-23. I would start by updating the firmware.

If that doesn’t solve it then, swap memory with the working computer.

If that doesn’t solve it then, remove all non-essential hardware cards.

If that doesn’t solve it then download the 5.15.x version of the linux kernel for FOG. Edit: I see now that you were at 5.15 and then upgraded. So this isn’t the issue.

[george1421]

@george1421 I did find this reference:

invalid opcode: 0000 [#1] PREEMPT SMP PTI

PTI(Page Table Isolation) is a Spectre/Meltdown correction at the bios and kernel level. Make sure they both are up to date.

ref: https://www.reddit.com/r/linuxquestions/comments/ajhw7e/what_can_cause_invalid_opcode_0000_1_preempt_smp/

So lets start with update the firmware.

[tlehrian]

@george1421 I thought I’d see if I could just install Windows on this computer without deploying an image to it. It gave me a BSOD. I did end up seeing the BIOS version was old, so I updated it to 01.08.08. Windows install BSOD again.

Memory and NVME hardware tests all come back OK. I have already tried swapping memory.

[tlehrian]

@george1421 So, I did update to BIOS version 01.08.08, but that still resulted in a Windows install BSOD:
IRQ_NOT_LESS_THAN_OR_EQUAL
followed by a reboot and another BSOD:
KMODE_EXCEPTION_NOT_HANDLED

Both of these point to either bad memory or device drivers. I’ll keep nosing around there.

[tlehrian]

@george1421 I replaced RAM with new, and now I get a BSOD: CLOCK_WATCHDOG_TIMEOUT which indicates it might be the processor.

This computer has been a thorn in our side - we have replaced the motherboard, and so it’s possible the processor didn’t survive the move. (or it went bad with the previous mobo).

I’m satisfied that this is not a FOG issue, but a hardware issue, so I think we can call this issue closed. Thanks for the help!

Tim

[george1421]

@tlehrian Ok so you get a BSOD with windows and under linux you get a kernel panic (same but different). So that points to a hardware issue with this computer.

I would swap memory with a known good computer, even if the mem test comes back OK. By moving parts hopefully the problem will move with the hardware.

Move other pcie attached devices like GPUs (if external) or other add in riser cards.

Now that you updated the bios, go in and reset the firmware settings back to factory defaults. In case there was a firmware setting change with the new firmware that corrects a known issue.

Sorry I can only give random ideas, but since this seems to be a hardware issues I can only make logical guesses here.