Posts made by george1421

@Bhav In addition to what Tom posted, the server response timeout is suspicious too. IF the target computer is in uefi mode and it downloaded undionly.kpxe the target computer should have responded with something about undionly.kpxe is not the right format. But in this case its getting a server response timeout. This makes me think two things. 1) your dhcp opition 66 is set incorrectly or the name of the file has a capitalization issue. For linux Undionly.kpxe and undionly.kpxe are not the same thing. Either way I would start with your dhcp server and make sure dhcp options 66 and 67 are set correctly for a uefi based computer.

@Quintin-Giesbrecht said in FOG Very Slow to Deploy Image - Lenovo Neo 50Q Gen 4:

Lenovo Neo 50Q Gen 4

I’m with Tom here, I would go with the latest FOS Linux kernel with this new hardware. You didn’t happen to mention what version of the FOS Linux kernel you are using (FOG UI -> FOG Configuration -> Kernel Update). I would go with the latest 6.6.x or later kernel. This is the easiest and quickest way to see if it solves the problem. It would be useful to know what version is your current kernel too so if we see this issue again we can help the next guy.

@lperoma Thank you for the pcaps. There is something strange going on here.

This is what I gleaned from our discussion (actors)

192.168.0.40 is dnsmasq
192.168.3.253 is my gateway (and DHCP server).
192.168.0.21 Fog Server
192.168.0.0/255.255.252.0 /23 network

When I look at the pcap the target computer 192.168.3.8 is downloading undionly.kpxe from the dnsmasq server 0.40 and not the fog server 0.21. At no time in the pcap do I see 0.21 in the dialog. Also it looks like the target computer is on a different subnet than the dhcp server and fog server. The pcap only shows one side of the conversation. I can see the Discover and Request bits of DORA. I’m missing what the dhcp server(s) are telling the target computer. This is because the client is on the other side of some router. The target computer is indeed a bios base system (HP) and the 0.40 server is running as a VM under proxmox.

So there are several questions here.

1. Is your fog server really 192.168.0.21
2. Why does the dnsmasq server have FOG’s pxe boot files? Those should be on the fog server.
3. Why does dnsmasq only tell the client computer to boot from itself and not the fog server?
4. You are capable of causing a good pxe boot and not good pxe boot. What are you doing different, or what is different between the two pcaps?

It would be interesting to see the other side (what the client is being told) by taking a third computer loaded with wireshark to the clients subnet and then use a capture filter of port 67 or port 68 to see what actors are involved and what the client is being told. At this time I’m only interested in what is not working.

ref:

@lperoma TBH I never look at the DORA process from the server side because it only tells us half the story. It looks like its working right in both flows.

Are you sure the dnsmasq is configured on the .40 server as a dhcp proxy or is it doing traditional dhcp functions. In a dhcp proxy config it only sends the boot file override command and then at the end of the DORA process it will then answer the dhcp proxy request on port 4011. Your dnsmasq configuration should look similar to the one in this tutorial: https://forums.fogproject.org/topic/12796/installing-dnsmasq-on-your-fog-server

When I try to debug this process I use the process outlined in this document: https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue

Since your dnsmasq server isn’t on your FOG server, I would run the tcpdump command from your dnsmasq server since there is some unicast communications to dnsmasq you would miss if you ran it on the fog server. It would be intersting to see what the client is being told on the pxe booting process that fails.

Minus the dhcp proxy communications you would miss you could also use wireshark with the capture filter of port 67 or port 68 running on a witness computer on the same IP subnet as the pxe booting target computer. There may be an unknown dhcp server mucking up the works on this pxe booting.

If you are willing to share the pcap I will look at it to see if there is anything that jumps out at me as the problem.

@LiamRetrams I can’t give you the exact steps to solve this but I can point you in a direction.

You will need to solve this in the windows realm. Once oobe starts FOG imaging is out of the picture so we can’t do anything in a post install script. You will need to use the bcdedit commands from inside windows to reset the boot order. You should be able to do this within the startupcomplete.cmd batch file or using the unattend.xml file in the first run section, or possibly deploy a fog snapin to run a batch file with the bcdedit commands.

I would manually reset the boot order to the way you want it then from within windows use the bcdedit commands to look at how it is ordered via bcdedit. That should give you the structure of what you need to do with the bcdedit commands to rebuild that format on a different system.

@lperoma said in Chainloading failed, hit 's' for the iPXE shell:

192.168.3.253

What device is that (manufacturer and model)? is that a soho router?

How does dnsmasq fit into this design? Is it setup as a proxy dhcp server?

The reason why I ask is I’ve seen many soho routers say (tell client computers) they are the boot server even if you have dhcp option 66 set correctly. The bios on the target computer is getting ipxe loaded correctly. Once iPXE starts it issues a dhcp discovery request again to find the boot server’s IP address. What it is getting this time is your gateway’s address.

With this new information I don’t think its spanning tree.

@lperoma So I can make the generalization statement that time fixes your issue?

What I want you to test is this.

PXE boot into the error and then press s to get to the ipxe console.

Wait 20 seconds.

Now key in the following at the console prompt
chain tftp://192.168.3.253/default.ipxe
Does this work and continue into the FOG iPXE menu. In that the only thing different between not working and working is time? (I’m driving to a point here).

If time fixes the issue, lets place a dumb switch (unmanaged cheap-o like monoprice) switch between the building network switch and the target computer. Now pxe boot the computer does it boot correctly? If yes then its possibly a spanning tree issue with your building network switch. Make sure your building switch has fastSTP, portfast or whatever your switch mfg calls the fast spanning tree protocol. I might be off base here but it sounds like it might be spanning tree. I know that processor is pretty old, but I don’t think its cpu related.

@lperoma Undionly.kpxe and snp.efi are probably the safest choices. Both of these use the built in driver built into the network adapter. The ipxe.pxe and ipxe.efi are more like the linux kernel where it has every known driver built into the boot loader. All of the other ipxe boot loaders are for unique situations that are not common.

So your system is an old 4th gen i5 processor. That’s strange that ipxe is misidentifying that CPU.

@lperoma said in PXE Boot Error - "Kernel Panic" Issue "XZ decompressor ran out of memory":

Any other idea? Can some one identify any issue with my dnsmasq ?

Its not a dnsmasq issue. dnsmasq only instructs to pickup ipxe.efi or undionly.kpxe once that boot loader is running dnsmasq has done its job. For some reason ipxe is misidentifying the processor as 32 bit only. Will you collect the exact processor model in these AIO?

I just saw in your previous message ipxe.kkpxe is being sent. Update your dnsmasq to send undionly.kpxe instead. That shouldn’t matter because ipxe.kkpxe uses the internal network adapters and undionly.kpxe uses the generic UNDI driver. But for consistency sake, lets make sure its not an ipxe issue.

@lperoma I don’t feel this is a pxe booting issue. There is only two flavors of iPXE. One for bios and one for uefi. Is this computer in bios or uefi mode?

The only way I can see this messing up because of dhcp is that the computer is in uefi mode, and you had something wrong with dnsmasq, where its loading the 32bit version of iPXE. That might falsely miss identify the processor as a 32 bit causing bzImage32 to be called.

The other thing I can think in the fog configuration -> fog settings. If you hit the expand all button then search for bzImage. If by chance someone changed the 64 bit fields to load bzImage32 and the 32 bit initrd. But that’s really unlikely.

Do all other computers work correctly except for this one?

@lperoma Looking at your screen shot a few things jump out at me.

Why is iPXE loading the 32 bit version of FOS Linux?

What hardware are you running here? What is the CPU? Is this super old hardware, like from the Pentium era?

I translate this error to the kernel (bzImage32) doesn’t have enough RAM memory to expand the FOS linux virtual hard drive (init_32.xz) into memory. If this IS a 32 bit system I still can’t understand why there is not enough ram for this kernel expansion. 32 bit arch is limited to 2GB of RAM. The FOS linux kernel and initrd compressed is < 400MB. Something is not adding up.

@cros I’m kind of spit balling here but your imgargs line especially around nfsroot doesn’t really follow what I expect.

FWIW here is what I have in my guide for debian 11. (I really don’t keep up with current distros anymore)

kernel tftp://${fog-ip}/os/debian/Server11.3/linux
initrd tftp://${fog-ip}/os/debian/Server11.3/initrd.gz
imgargs linux initrd=initrd.gz root=/dev/nfs boot=casper netboot=nfs nfsroot=${fog-ip}:/images/os/debian/Server11.3/ locale=en_US.UTF-8 keyboard-configuration/layoutcode=us quiet splash ip=dhcp rw
boot || goto MENU

For me your imgargs line stands out as is your 192.168.7.5 server your fog server or a different server? If its your fog server you can make the line a bit more portable by using ${fog-ip} which will be replaced by the IP address of the fog server by iPXE. Secondly did you create an /nfs share on your fog server because that is not one of FOGs standard nfs shares. On the fog server you should be able to run the command showmount -e 127.0.0.1 to see the list of nfs shares on the fog server. In my imgarg command you can see I’m using /images/os which is in the path of /images on the fog server and /images is an NFS share.

ref: https://forums.fogproject.org/post/150256

@jcarr ok that is an x86 system. I see its strange that the kernel doesn’t try to boot at all. That’s not a new system (8th Gen) there shouldn’t be an issue with FOS Linux starting up. I understand that all actions from the ipxe menu doesn’t boot they all rely on FOS Linux kernel starting up. Have you tested an older kernel, maybe either in the 5.x or 4.x (last choice) range?

Is the issue with just this computer or all computers you try to pxe boot?

If its just this computer, is the firmware up to date?

I have to shot gun this, because it should boot.

@jcarr Just for clarity this target system is x86 based?

Also in the fog configuration ->fog settings panel of the ui set the logging level to 7. It may be either 4 or 1 by default. This should print more information on the boot screen. What its showing so far is that its working.

@rodluz Good going. I also saw the issues with the lenovo’s ARM notebooks. I think we may need to wait until 6.12 is released. Yeah, without networking or keyboard you are kind of stuck. I was thinking that we could just ssh into the fos engine to bypass the keyboard issue, but then no network and no way to set root’s password. That would have to be done using buildroot to assign a default password to root. Not worth the effort until you get keyboard access. I just has a flash of an idea of a serial console, but that system probably doesn’t have a real serial port either.

@PatrickL You might want to review this post: https://forums.fogproject.org/topic/17648/massive-cpu-usage-from-a-service

That .sysvd is suspicious and very similar to the thread above.

@jfernandz said in BitLocker compatibility:

The TPM point is a good one, but … almost all machines we work with have an “easily” accessible/replaceable TPM hardware module, could just we restore some disk image in a new machine with the TPM of the old one? Would this work?

-Or- just decrypt your golden/mother image before image capture, then either have the unattend.xml or gpo policy encrypt the drive when it hits the target computer hardware? Don’t make it harder on yourself than needed. I’m sure your users are willing to do that to you for free.

@Eliseu What I see here is that ipxe is booting, but this isn’t FOG’s version of iPXE. Are you running this pxe booting computer in virtual box?

@stokehall Just a quick oracular review and it will eventually ship with linux kernel 6.11, and it kind of works with the latest ARM processors. But we are getting close and kernel 6.11 is probably a good place to start. We might want to start with the default config for the qualcom processors and then add in the bits that FOG needs. Linux kernel 6.11 was released on 15-Sep.

@jfernandz Actually bitlocker fde (full disk encryption) was developed to prevent what you are trying to do. I don’t remember if the developers put a stop point in the code if fde is detected but technically FOG will copy a bitlocker protected disk, but it will do it in raw mode. The issue you will have if fog cloned the disk image is that bitlocker encrypts the disk with a key that is held in the TPM chip. So even if FOG cloned the disk, the data would not be able to be used because the TPM keys would not match. This prevents cloning or accessing data on protected media.
For the data to be cloned and usable afterwards you must decrypt the drive before cloning.