TFTP Open Timeout on New Fog Install
-
@fhrivers So pxe booting worked OK with 0.3x version of fog and you only updated (installed) 1.5.5 and have pxe booting issues? Was there any gap in time between you using fog 0.3x and 1.5.5?
-
@fhrivers said in TFTP Open Timeout on New Fog Install:
Option 67 is set to undionly.kpxe like it’s supposed to
Not for UEFI booting machines though. If your machines are UEFI booting then you need to use ipxe.efi. Details on this see here: https://wiki.fogproject.org/wiki/index.php?title=BIOS_and_UEFI_Co-Existence
I guess we need more details to be able to help:
- Which Linux OS do you use? CentOS? Debian? Ubuntu? Did you disable SELinux and the firewall?
- Is TFTP running? Command to check:
netstat -antup | grep ":69"orss -antul | grep ":69"(if netstat is not installed) - Do you have other machines (beside Lenovo X1 Carbon and my VMWare Workstation VM which can both be tricky) that do properly PXE boot on the new FOG server?
- Take a picture of the actual TFTP error on screen an post here!
- Have you read the troubleshooting guide in our wiki? https://wiki.fogproject.org/wiki/index.php?title=Tftp_timeout… (please follow the tests to see if you can manually download the iPXE binary file)
-
@george1421 3 years and one company! LOL. Also did that in a VMWare environment. Lots of differences this time!
-
@Sebastian-Roth Netstat returns this:
udp 0 0 0.0.0.0:69 0.0.0.0:* 3332/xinetd
-
-
Okay, sorry about the spam as I’m troubleshooting in parallel with providing you with more information. I’m running Fog in a CentOS VM running in Virtualbox which is running on a Windows 10 host. So I may be running into Windows firewall issues. I cannot ping external machines on the network from CentOS.
-
@fhrivers Take a look at this wiki article:
https://wiki.fogproject.org/wiki/index.php?title=CentOS_7#Continue_pre-configAs well I am wondering how you setup the networking of the VirtualBox VM. Best if you can take a picture of the network settings in VirtualBox and post here.
-
VirtualBox is running on a Windows 10 PC and is setup in Bridge mode. I had some networking issues due to an aggressive web filter on our network, but other than that, this has been functioning normally.
Please disregard the fact that I couldn’t ping anything from the server. I can’t ping them from other machines from the host either. So I’m not pursuing that path right now.
I did view the TFTP troubleshooting page and I checked everything there with no success.
-
Just for good measure, I tore out the VirtualBox solution and stood FOG up on an old desktop. Same error.
I even ran tftp -i <ip> get udionly.kpxe
connect request failed
Same problem with legacy BIOS and UEFI. I can ping the FOG server from my client machine, but FOG can’t ping client.
-
@fhrivers Run
iptables -L -n -vto see if the local firewall on the Linux system is enabled. Take a picture and post here. -
@Sebastian-Roth Attached is output.
-
@fhrivers Please try the following:
iptables -L -n -v | grep "dpt:69"Now you see the first number in that line. That’s how often this rules has been used so far. In the output you posted this was 26 times and that means the initial TFTP connection is going through. But that is not enough. Find the output of a network packet capture below. You see first DHCP handshake (four packets) and then TFTP:
19:42:57.319383 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 08:00:27:ab:0f:cc, length 548 19:42:58.327617 IP 192.168.2.7.67 > 255.255.255.255.68: BOOTP/DHCP, Reply, length 300 19:42:59.340794 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 08:00:27:ab:0f:cc, length 548 19:42:59.355690 IP 192.168.2.7.67 > 255.255.255.255.68: BOOTP/DHCP, Reply, length 300 19:42:59.358575 IP 192.168.2.10.2070 > 192.168.2.7.69: 31 RRQ "undionly.kkpxe" octet tsize 0 19:42:59.378177 IP 192.168.2.7.32788 > 192.168.2.10.2070: UDP, length 14 19:42:59.378538 IP 192.168.2.10.2070 > 192.168.2.7.32788: UDP, length 17 19:42:59.379556 IP 192.168.2.10.2071 > 192.168.2.7.69: 36 RRQ "undionly.kkpxe" octet blksize 1456 19:42:59.381209 IP 192.168.2.7.60001 > 192.168.2.10.2071: UDP, length 15 19:42:59.381310 IP 192.168.2.10.2071 > 192.168.2.7.60001: UDP, length 4 19:42:59.381392 IP 192.168.2.7.60001 > 192.168.2.10.2071: UDP, length 1460 19:42:59.381717 IP 192.168.2.10.2071 > 192.168.2.7.60001: UDP, length 4 19:42:59.382154 IP 192.168.2.7.60001 > 192.168.2.10.2071: UDP, length 1460 19:42:59.382273 IP 192.168.2.10.2071 > 192.168.2.7.60001: UDP, length 4 ...While the first TFTP packet goes to UDP port 69 for the actual transfer of the file random high ports are being used. This is where your clients timeout I am fairly sure!
For now I would suggest you disable the firewall on your FOG server to see if I am on the right track. If that works then you might start reading more about Linux firewalling and how to enable it the way to still be able to use FOG. Just a hint on that: FOG uses NFS and FTP beside TFTP, which all use random ports. Therefore we usually tend to leave the firewall disabled anyway.
As well you might want to take a look at SELinux, as it can cause issues as well: https://linuxize.com/post/how-to-disable-selinux-on-centos-7/
-
@Sebastian-Roth Same error in Windows tftp test with firewall disabled. Its not an access denied so I’m fairly confident its not a firewall issue. I’m getting this on a VM and physical hardware install of CentOS 7.
Very strange. I even rebooted after the change for good measure.
-
@fhrivers Is your fog server, dhcp server, and pxe boot client all on the same subnet? If so lets grab a pcap of that pxe boot process. There is something going sideways here that we don’t expect.
https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue
Upload the pcap to a google drive or dropbox and share the link as public. Post the link here and we will take a look at it. I recommend doing it this way because then YOU have control of the file’s existance after the debugging session is done.
Also if you install the tftp client feature in your windows computer, can you use the tftp get command to download the undionly.kpxe boot file from your FOG server? You may need to temporarily disable the windows firewall for the tftp client command to work correctly.
-
@george1421 Edit: Misread the question.
Everything is on the same subnet. We’re on a .23 subnet. In fact all the devices I’m using for testing are plugged into the same switch.
I’ll work on getting the info you need.
-
@fhrivers said in TFTP Open Timeout on New Fog Install:
@george1421 Fog is not handling DHCP, our corporate router does that. I’ll look at providing you that info.
As long as they are all on the same vlan (subnet) then we can get an accurate picture of what the target computer is being told.
-
@fhrivers said in TFTP Open Timeout on New Fog Install:
Very strange. I even rebooted after the change for good measure.
Which change did you do exactly?? Disabled the firewall as suggested? I am not talking about the Windows firewall here!
Make sure firewall rules are gone after the disabled:
iptables -L -n -v Chain INPUT (policy ACCEPT 70204 packets, 115M bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 22073 packets, 26M bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 64101 packets, 8327K bytes) pkts bytes target prot opt in out source destinationAll three default chains are empty and default policy set to
ACCEPT. -
@Sebastian-Roth I disabled firewalld in CentOS.
-
@Sebastian-Roth Here’s the output of my iptables:
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destinationChain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destinationChain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination -
@george1421 Where is the output.pcap file saved?
