PXE partial success, no tftp

[thezman007]

Greetings and thanks for the help.

My current setup seems to allow our PXE boot to partially work, but ultimately fails. It appears that our proxyDHCP via dnsmasq is working and our main DHCP server is handing out IPs while our fog server is directing devices to itself for PXE services, but the overall process fails once tftp should be serving the .efi file. We’ve tried using a different computer when attempting to PXE to try and eliminate model specific quirks. I’ve also tried changing the file dnsmasq should serve (snponly.efi or ipxe.efi) with no change. tftp via locahost works as expected, tftp over LAN fails. There are NO tftp requests seen from tcpdump during PXE boot, but I can’t provide that data until my tech returns on-site next week.

I have setup our fog server by installing a fresh version of Ubuntu and pulling the latest stable version from github then running the install.sh. This is in a small office with Ubiquiti switching. DHCP snooping is off, VLANs are not in use. BIOS settings include: Secure boot is off, UEFI network stack is enabled using ipv4. I can provide more details but didn’t want to overload the first posting.

Details

OS and kernel

Linux STL-FOGBUNTU 6.14.0-37-generic #37~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 20 10:25:38 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
Distributor ID: Ubuntu
Description: Ubuntu 24.04.3 LTS
Release: 24.04
Codename: noble

Fog version

1.5.10.1734

Services

systemctl status dnsmasq --no-pager -l
● dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server
Loaded: loaded (/usr/lib/systemd/system/dnsmasq.service; enabled; preset: enabled)
Active: active (running) since Fri 2026-01-23 15:58:19 CST; 1min 11s ago
Process: 3886 ExecStartPre=/usr/share/dnsmasq/systemd-helper checkconfig (code=exited, status=0/SUCCESS)
Process: 3891 ExecStart=/usr/share/dnsmasq/systemd-helper exec (code=exited, status=0/SUCCESS)
Process: 3898 ExecStartPost=/usr/share/dnsmasq/systemd-helper start-resolvconf (code=exited, status=0/SUCCESS)
Main PID: 3897 (dnsmasq)
Tasks: 1 (limit: 18571)
Memory: 904.0K (peak: 3.9M)
CPU: 72ms
CGroup: /system.slice/dnsmasq.service
└─3897 /usr/sbin/dnsmasq -x /run/dnsmasq/dnsmasq.pid -u dnsmasq -r /run/dnsmasq/resolv.conf -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new --local-service --trust-anchor=.,20326,8,2,E06D44B80B8F1D39A95C0B0D7C65D08458E880409BBC683457104237C7F8EC8D --trust-anchor=.,38696,8,2,683D2D0ACB8C9B712A1948B27F741219298D0A450D612C483AF444A4C0FB2B16

Jan 23 15:58:19 STL-FOGBUNTU dnsmasq[3897]: started, version 2.90 DNS disabled
Jan 23 15:58:19 STL-FOGBUNTU dnsmasq[3897]: compile time options: IPv6 GNU-getopt DBus no-UBus i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset nftset auth cryptohash DNSSEC loop-detect inotify dumpfile
Jan 23 15:58:19 STL-FOGBUNTU dnsmasq-dhcp[3897]: DHCP, proxy on subnet 192.168.2.0
Jan 23 15:58:19 STL-FOGBUNTU dnsmasq-dhcp[3897]: DHCP, proxy on subnet 192.168.2.0
Jan 23 15:58:19 STL-FOGBUNTU dnsmasq-dhcp[3897]: DHCP, sockets bound exclusively to interface enp0s31f6
Jan 23 15:58:19 STL-FOGBUNTU resolvconf[3908]: Dropped protocol specifier ‘.dnsmasq’ from ‘lo.dnsmasq’. Using ‘lo’ (ifindex=1).
Jan 23 15:58:19 STL-FOGBUNTU resolvconf[3908]: Failed to set DNS configuration: Unit dbus-org.freedesktop.network1.service not found.
Jan 23 15:58:19 STL-FOGBUNTU systemd[1]: Started dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server.
Jan 23 15:59:22 STL-FOGBUNTU dnsmasq-dhcp[3897]: 4276486162 available DHCP subnet: 192.168.2.0/255.255.255.0
Jan 23 15:59:22 STL-FOGBUNTU dnsmasq-dhcp[3897]: 4276486162 available DHCP subnet: 192.168.2.0/255.255.255.0

systemctl status tftpd-hpa -l --no-pager
● tftpd-hpa.service - LSB: HPA’s tftp server
Loaded: loaded (/etc/init.d/tftpd-hpa; generated)
Active: active (running) since Fri 2026-01-23 15:51:28 CST; 10min ago
Docs: man:systemd-sysv-generator(8)
Process: 2669 ExecStart=/etc/init.d/tftpd-hpa start (code=exited, status=0/SUCCESS)
Tasks: 1 (limit: 18571)
Memory: 724.0K (peak: 1.9M)
CPU: 19ms
CGroup: /system.slice/tftpd-hpa.service
└─2698 /usr/sbin/in.tftpd --listen --user tftp --address 0.0.0.0:69 --secure --ipv4 --create --permissive -vv --umask 027 /var/lib/tftpboot

Jan 23 15:51:28 STL-FOGBUNTU systemd[1]: Starting tftpd-hpa.service - LSB: HPA’s tftp server…
Jan 23 15:51:28 STL-FOGBUNTU tftpd-hpa[2669]: * Starting HPA’s tftpd in.tftpd
Jan 23 15:51:28 STL-FOGBUNTU tftpd-hpa[2669]: …done.
Jan 23 15:51:28 STL-FOGBUNTU systemd[1]: Started tftpd-hpa.service - LSB: HPA’s tftp server.

Listening ports

ss -lunp | egrep ‘:(67|69|4011)\b’
UNCONN 0 0 0.0.0.0%enp0s31f6:67 0.0.0.0:* users:((“dnsmasq”,pid=3897,fd=4))
UNCONN 0 0 0.0.0.0:69 0.0.0.0:* users:((“in.tftpd”,pid=2698,fd=4))
UNCONN 0 0 0.0.0.0%enp0s31f6:4011 0.0.0.0:* users:((“dnsmasq”,pid=3897,fd=5))

tftp config

tftp daemon config
cat /etc/default/tftpd-hpa
# /etc/default/tftpd-hpa
# FOG Modified version
TFTP_USERNAME=“tftp”
TFTP_DIRECTORY=“/var/lib/tftpboot”
TFTP_ADDRESS=“0.0.0.0:69”
TFTP_OPTIONS=“–secure --ipv4 --create --permissive -vv --umask 027”

Permissions

ls -ld /tftpboot
drwxrwxr-x 5 tftp tftp 4096 Jan 14 11:44 /tftpboot

tftp testing

tftp test via localhost - SUCCESS
root@STL-FOGBUNTU:/test# ls
root@STL-FOGBUNTU:/test# tftp localhost
tftp> get snponly.efi
tftp> quit
root@STL-FOGBUNTU:/test# ls
snponly.efi

No traffic is seen via tcpdump

tftp test via Windows host over LAN - FAIL
PS C:\temp> tftp 192.168.2.231 get snponly.efi
Connect request failed

tcpdump during failed test
tcpdump -ni enp0s31f6 udp port 69
tcpdump: verbose output suppressed, use -v[v]… for full protocol decode
listening on enp0s31f6, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:10:23.293038 IP 192.168.2.226.49370 > 192.168.2.231.69: TFTP, length 23, RRQ “snponly.efi” netascii
16:10:24.298517 IP 192.168.2.226.49370 > 192.168.2.231.69: TFTP, length 23, RRQ “snponly.efi” netascii
16:10:26.311059 IP 192.168.2.226.49370 > 192.168.2.231.69: TFTP, length 23, RRQ “snponly.efi” netascii
16:10:30.320719 IP 192.168.2.226.49370 > 192.168.2.231.69: TFTP, length 23, RRQ “snponly.efi” netascii
16:10:38.333886 IP 192.168.2.226.49370 > 192.168.2.231.69: TFTP, length 23, RRQ “snponly.efi” netascii
16:10:46.343141 IP 192.168.2.226.49370 > 192.168.2.231.69: TFTP, length 23, RRQ “snponly.efi” netascii
16:10:54.346973 IP 192.168.2.226.49370 > 192.168.2.231.69: TFTP, length 23, RRQ “snponly.efi” netascii
16:11:02.355580 IP 192.168.2.226.49370 > 192.168.2.231.69: TFTP, length 23, RRQ “snponly.efi” netascii
16:11:10.374724 IP 192.168.2.226.49370 > 192.168.2.231.69: TFTP, length 23, ERROR EUNDEF “timeout on receive”

[george1421]

@thezman007 said in PXE partial success, no tftp:

My current setup seems to allow our PXE boot to partially work, but ultimately fails. It appears that our proxyDHCP via dnsmasq is working and our main DHCP server is handing out IPs while our fog server is directing devices to itself for PXE services, but the overall process fails once tftp should be serving the .efi file. We’ve tried using a different computer when attempting to PXE to try and eliminate model specific quirks. I’ve also tried changing the file dnsmasq should serve (snponly.efi or ipxe.efi) with no change. tftp via locahost works as expected, tftp over LAN fails. There are NO tftp requests seen from tcpdump during PXE boot, but I can’t provide that data until my tech returns on-site next week.

This is the most important section.

what I want you to do is run tcpdump from the fog server. I want you to use the pcap filter of port 67 or port 68 or port 4011 or port 69

That will capture dhcp, proxy-dhcp and tftp.

ref: https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue?_=1769224516191

Review the pcap with wireshark. You should see the DORA process if the fog server is on the same subnet as the pxe booting client.

Discover
Offer
Request
Ack/Nack

What will be important to watch is to make sure the client is getting two offer packets. Once will be from your main dhcp server and the second one from dnsmasq. If you are not seeing the one from dnsmasq server then that is the start of the problem. If you do see two and one is from your dnsmasq server then go to the next part.

Now that you verified that dnsmasq is seeing the DISCOVER packet and responded with an OFFER packet then after DORA you should see the client call back to dnsmasq on port 4011. In that transaction the client will be told the boot server and boot file. Verify these are correct.

And finally the client should reach out to the FOG server over tftp to first request the file size then request the file. So there will be two tftp communications, then the file should download.