Posts made by Sebastian Roth

@george1421 said in connection timed out chainloading failed:

Well then the FOG iPXE script is at fault (not laying actual blame here). It is responsible for taking over once iPXE starts. We may need to look at this exception case here to see if there is a weakness in the script.

Good point. Now that I think about it a little more I have a faint memory of trying to detect such a situation in the iPXE script but failed. Though I don’t remember the details but it might be iPXE just not using the information from the proxy DHCP in case it receives next server from the other one. Not sure though.

@george1421 said in connection timed out chainloading failed:

dnsmasq (proxydhcp) should override any settings for both bootp and dhcp pxe booting.

It kind of does as we see in the screenshot that PXE ROM is actually able to find and load the iPXE binary. Though it seems like iPXE is getting confused by the clearOS dnsmasq (not in proxy mode) sending “Next Server IP” in the DHCP body and therefore wants to load default.ipxe from the wrong IP.

One check is to see if at the end of the dora process the target computer reaches out to the fog server on udp port 4011. If that dialog is not happening then the target computer will go with what is in the OFFER packet.

Yes it does on the first DORA round (PXE ROM) but not on the second round (iPXE).

@mstabrin said in Tasktypeedit setup:

FullWipe - Reboot

Do you mean scheduling a FullWipe task and that should do a reboot of the host and boot into the FullWipe? While I have not use wiping in a long time I would think that a scheduled task (deploy, capture, wipe, …) should actually reboot a host if fog-client is installed and working properly.

@choppaholic26 Ok, but after that we should also see the requests for http://192.168.70.11/fog//index.php in that access log! Ignore the double slash in the URL, it looks ugly but Apache can handle that.

So as a next step I would suggest you manually create the VirtualBox VM as a host in the FOG web UI (using its MAC address) and schedule a debug deploy task for it. I know there is nothing to deploy but this way we can get to a debug command shell.

In the FOG web UI click on the host you just manually created -> Basic Tasks tab -> Deploy -> enable checkbox for debug task and create the task. Now PXE boot the VirtualBox VM as usual. This time you should not get the menu but it should boot straight up to the last issue. With debug enabled you should get to a command shell after a while. Please run the following commands and take a picture:

ip a s
echo "${web}"
curl -Ik "${web}"/index.php

Note: The first curl parameter is an I as in India, not l or 1. If that call fails for a non obvious reason you might add the verbose switch to get more information curl -Ikv ...

By the way, did you see my post on extending your dhcpd.conf to get option 66/67? https://forums.fogproject.org/post/141631

@choppaholic26 said in Another TFTP timeout issue:

I PXE booted to the failed registration attempt but unless I’m doing it wrong this file is completely empty for me.

Sorry my bad! Newer Ubuntu versions seem to log those requests to /var/log/apache2/other_vhosts_access.log…

@jcyrus_rti You are welcome!

Sorry if my answer sounded harsh. Didn’t mean to.

@jbandy What output do you get from df -h. It could be a cause of disk being full causing such strange things.

@mstabrin You might want to look into using ethtool to enable Wake-on-LAN within FOS before you shutdown the system.

https://wiki.archlinux.org/index.php/Wake-on-LAN#Enable_WoL_on_the_network_adapter

For testing I suggest you schedule a debug deploy task and boot up that machine you just tasked. You should end up in a command shell after boot up (and hitting ENTER twice). Now see what ethtool is telling you and if you can enable WOL. Probably depends on the NIC and driver.

Then some drivers seem to need parameters to enable WOL, like “3c59x enable_wol=1”…

Beside this there seem to be other things that could prevent Wake-on-LAN from happening. Some articles talk about /proc/acpi/wakeup but we don’t seem to have this enabled in our kernel. So if ethtool doesn’t help we might give you instructions on how to build your own kernel and get /proc/acpi/wakeup to play with this.

@Dungody Thanks for the update and pictures! Definitely looks like you deploy to the NVMe driver. I still can’t see why a USB key plugged into the ProBook would make a difference. Though it sounds like it does.

In the picture I notice that you have partclone version 0.2.89. In FOG 1.5.9 we use the newer partclone 0.3.13. So I am wondering which version of FOG you use and if that is also playing a role here.

@Jacob-Gallant Would you want to give that a try as well?

@jcyrus_rti Please be more specific on what you mean by it doesn’t work!

As far as I can tell you can push both UEFI and legacy BIOS images on either more using FOG (people do so all the time) but you can’t usually boot that deployed image on the machine. That is not something FOG is doing wrong. It’s just UEFI and legacy BIOS being so different and you can’t just deploy one and hope to boot it up in the other mode like that.

Yes you need different images for UEFI and legacy BIOS!

Had a look at the PCAP and we clearly see the clearOS dnsmasq sending next server information in the DHCP offer/ack packet body. I think it shouldn’t do that in this setup because it’s confusing clients that receive the IP from DHCP and PXE boot information from the proxy DHCP.

Maybe this is just the default behavior of dnsmasq even if it’s not setup as DHCP proxy or full PXE bootp service. Not sure.

@geardog In that case we’ll probably need to look at the actual packets on the wire: https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue

@choppaholic26 The new pictures you posted clearly show that it does get an IP from the DHCP. So from the message “Either DHCP failed or we were unable to access http://192.168.70.11/…” I would imagine the later one to fail. Though the URL looks all fine.

Can you please run tail -f /var/log/apache2/access.log while PXE booting the VirtualBox VM? You should see requests for boot.php, then bzImage and init.xz and lastly another request as connection check which we see in the picture. Do you see all those?

@choppaholic26 said in Another TFTP timeout issue:

However, if I wait a couple of seconds while in the FOG menu, the registration will start (I tested waiting a few seconds on memtest and it started running fine) but DHCP will fail.

Well, that’s awkward. Maybe some rate limiting firewall on the Proxmox host?! Just a wild guess.

As far as the setup, I have my physical proxmox server with the installed FOG VM setup to use its interfaces. I have a laptop with VirtualBox running that is plugged directly into the back of the server. To PXE boot I have a bridged adapter for VirtualBox to use to PXE in the VirtualBox settings.

Ok, sounds reasonable. I do understand that using VirtualBox to test can make things easier and in this case it really is of help to see that TFTP actually works.

Edit: Thought I should clarify though, that the TFTP timeout issue when PXE booting the actual laptop hasn’t changed.

Ok. As mentioned before some PXE booting devices are happy to use whatever information they get while others are more picky and won’t boot unless you provide exactly what they are after. So from what we’ve seen so far it looks like the physical notebook doesn’t like the missing DHCP options (66/67).

I have to admit that I haven’t looked at the DHCP offer/ack packets that isc-dhcp sends out in a long time. Seems like we actually miss the options 66 and 67 in our default config. Probably because 99.9% of machines properly PXE boot with the information given in the DHCP body as mentioned below.

To get the extra options you need to adjust your config and add option tftp-server-name and option bootfile-name to make it look like this:

...
    next-server 192.168.70.11;
    option tftp-server-name "192.168.70.11";
    class "Legacy" {
        match if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:00000";
        filename "undionly.kkpxe";
        option bootfile-name "undionly.kkpxe";
    }
...

@geardog Looks a bit like the actual DHCP server also hands out some PXE booting information that would overwrite the information you send via dnsmasq DHCP proxy.

So question is, what kind of DHCP server is handing out the IPs.

@choppaholic26 Wow, seems like an endeavor and I am not exactly sure where to jive in. Although George is right the DHCP packets seem to be missing the actual DHCP options 66 and 67 I still think that some clients can handle this and PXE boot using the information seen in the DHCP base body.

I would focus on capturing more traffic on the FOG server to see if the TFTP requests actually make it to the server. If you have used the command as suggested in the post by George (tcpdump -w output.pcap port 67 or port 68 or port 69 or port 4011) we should actually see TFTP requests coming in (as from the initial picture you posted we know it tries to).

You might check if the Ubuntu firewall is blocking it! sudo ufw disable and try again.

By the way, where did you setup the VirtualBox stuff so it would PXE boot off you FOG server running in a VM on the Proxmox server. I am wondering if it’s more due to complexity in the setup that things don’t work. On the other hand the new screenshots show it actually pull the iPXE binary from the server (so TFTP transfer must work). Beside booting into the memdisk task, have you tried doing a registration through the FOG menu yet?

We seem to have a few people having issues with that particular NIC.
https://forums.fogproject.org/topic/15141/optiplex-5480-all-in-one
https://forums.fogproject.org/topic/14878/dell-5080-won-t-image
https://forums.fogproject.org/topic/15038/dell-optiplex-5080-network-interface-not-found
https://forums.fogproject.org/topic/14875/lenovo-thinkpad-p15-t15-no-network-interface-found-kernel-might-not-have-the-correct-driver

While some were able to make it work updating to the 5.6.18 kernel others still have issues to get it to work.

@jcyrus_rti First let’s make sure it actually boots the newer kernels - see my below post to check.

@jcyrus_rti As George said, don’t mix up FOG server OS kernel and the FOS (FOG OS booting to do the work) kernel. The later one will be used on PXE booting the machines.

Now it sounds like you have updated to 5.10.x already. Please double check if that is the case by running the following commands and post output here:

file /var/www/html/fog/service/ipxe/bzImage*
file /var/www/fog/service/ipxe/bzImage*

Yeah, switching to a different iPXE binary can help in some cases too.

Though I want to bring up what I have posted two weeks ago as well: https://forums.fogproject.org/post/141320 (compiling the latest version of iPXE)

@EZY4 You can try using a different iPXE binary like snponly.efi at first. This might work with the A3510. Though on the other hand you can’t be sure if this will work for all your other devices.

Not sure if you can setup a particular DHCP policy based on the MAC address just for those machines to use snponly.efi…

Other than that I suggest you build the iPXE binaries from the latest official code base and see if those work on the A3510. Assuming you have the FOG stuff in /root/fogproject try this:

sudo -i
cd /root/fogproject/utils/FOGiPXE/
./buildipxe.sh
cp ../../packages/tftp/ipxe.efi /tftpboot/

Pay attention to the long output when running ./buildipxe.sh as it might fail for whatever reason. If you feel it ends with an error, post the last 20 lines of the output here and we will have a look.

Make sure ipxe.efi is set in your DHCP again, boot up a machine and pay attention to where the preemble tells you the version number iPXE (g…). That should be different to g4bd0 (default on 1.5.9 installs).