PXE-E32: TFTP open time out on palo alto dhcp server

fernando.martinez

Hi Sebastian,

Thanks for your reply.
The fog version is: fogproject-1.5.7

Option 66:

Option 67:

This is the error:

Thanks for your help.

Sebastian Roth

@fernando-martinez One first thing I noticed is the subnet mask not being standard. While this is totally fine if intended (to get a bigger subnet of 192.168.96.1 - 192.168.97.254) I still want to point that out just in case.

Second I notice the “Option Type” setting for option 66. While setting this to “IP Address” does kind of make sense I am not sure what exactly this does. Have you tried setting it to “ASCII” as well yet?

I suppose 192.168.96.205 is your FOG server, right? Please run the following command on the FOG server and post output here: netstat -antup | grep ":69" (possibly you need to install package net-tools or similar depending on the Linux OS you use)

fernando.martinez

@Sebastian-Roth said in PXE-E32: TFTP open time out on palo alto dhcp server:

netstat -antup | grep “:69”

Hi Sebastian,

Yes, my fog server is 192.168.96.204. I try to put option 66 in “ascii” and i have the same problem.

Thanks again.

Sebastian Roth

@fernando-martinez Service seems to be running. Maybe firewall?? On the FOG server run iptables -L -n -v and firewall-cmd --state - post output here.

Or possibly some other firewall in your network between clients and the FOG server. Though this would be quite unusual with both being in the same subnet.

fernando.martinez

@Sebastian-Roth said in PXE-E32: TFTP open time out on palo alto dhcp server:

@fernando-martinez Service seems to be running. Maybe firewall?? On the FOG server run iptables -L -n -v and firewall-cmd --state - post output here.
Or possibly some other firewall in your network between clients and the FOG server. Though this would be quite unusual with both being in the same subnet.

This is the output:

Sebastian Roth

@fernando-martinez Is there a router/gateway between the clients and the FOG server?

If not I might suggest you turn off the PXE information on your Palo Alto DHCP server for now and try using dnsmasq on your FOG server. This is not a replacement of the whole DHCP server but just a so called DHCP Proxy which adds the PXE boot information. Technically it’s just sending another DHCP answer to the clients with no IP address information but only the PXE info. This works quite well in a lot of cases. See information in this here: https://forums.fogproject.org/topic/12796/installing-dnsmasq-on-your-fog-server

fernando.martinez

Hi Sebastian

This sounds good, I tried it and the error is different now.

It seems it can’t find the pxe file.
The dnsmasq configuration shouldn’t give me problems with my dns server, really?

Many thanks!

Sebastian Roth

@fernando-martinez Please run ls -al /tftpboot/ on your FOG server and post output here.

george1421

@fernando-martinez will you post the dnsmasq config file as you have it. DNSMASQ is getting to this line and then either not sending the right command or the file name is for what ever reason incorrectly formatted.

pxe-prompt="Booting FOG Client", 1

Also if you go into the /var/log directory on the fog server and key in grep -R -i tftp * that will show you the file and what the target computer is asking for via tftp. Just look in that list for the IP address of the target computer 192.168.96.61

fernando.martinez

@Sebastian-Roth said in PXE-E32: TFTP open time out on palo alto dhcp server:

ls -al /tftpboot/

root@SRVOPTIFOGADMIN:~# ls -al /tftpboot/
total 7264
drwxrwxrwx  5 fogproject root    4096 feb  1 12:54 .
drwxr-xr-x 26 root       root    4096 ene 28 13:14 ..
drwxrwxrwx  2 fogproject root    4096 ene 28 13:14 10secdelay
-rwxrwxrwx  1 fogproject root     868 ene 28 13:14 boot.txt
-rwxrwxrwx  1 root       root     459 ene 28 13:14 default.ipxe
drwxrwxrwx  2 fogproject root    4096 ene 28 13:14 i386-efi
-rwxrwxrwx  1 fogproject root  226944 ene 28 13:14 intel.efi
-rwxrwxrwx  1 fogproject root   98224 ene 28 13:14 intel.kkpxe
-rwxrwxrwx  1 fogproject root   98272 ene 28 13:14 intel.kpxe
-rwxrwxrwx  1 fogproject root   98356 ene 28 13:14 intel.pxe
-rwxrwxrwx  1 fogproject root 1016960 ene 28 13:14 ipxe.efi
-rwxrwxrwx  1 fogproject root  880640 ene 28 13:14 ipxe.iso
-rwxrwxrwx  1 fogproject root  359745 ene 28 13:14 ipxe.kkpxe
-rwxrwxrwx  1 fogproject root  359793 ene 28 13:14 ipxe.kpxe
-rwxrwxrwx  1 fogproject root  359243 ene 28 13:14 ipxe.krn
-rwxrwxrwx  1 fogproject root  359243 ene 28 13:14 ipxe.lkrn
-rwxrwxrwx  1 fogproject root  359688 ene 28 13:14 ipxe.pxe
-rwxrwxrwx  1 fogproject root 1409024 ene 28 13:14 ipxe.usb
-rwxrwxrwx  1 fogproject root  123448 ene 28 13:14 ldlinux.c32
-rwxrwxrwx  1 fogproject root  187820 ene 28 13:14 libcom32.c32
-rwxrwxrwx  1 fogproject root   26468 ene 28 13:14 libutil.c32
-rwxrwxrwx  1 fogproject root   26140 ene 28 13:14 memdisk
-rwxrwxrwx  1 fogproject root   29208 ene 28 13:14 menu.c32
-rwxrwxrwx  1 fogproject root   43210 ene 28 13:14 pxelinux.0.old
drwxrwxrwx  2 fogproject root    4096 ene 28 13:14 pxelinux.cfg
-rwxrwxrwx  1 fogproject root  225856 ene 28 13:14 realtek.efi
-rwxrwxrwx  1 fogproject root   99051 ene 28 13:14 realtek.kkpxe
-rwxrwxrwx  1 fogproject root   99099 ene 28 13:14 realtek.kpxe
-rwxrwxrwx  1 fogproject root   99119 ene 28 13:14 realtek.pxe
-rwxrwxrwx  1 fogproject root  225152 ene 28 13:14 snp.efi
-rwxrwxrwx  1 fogproject root  225440 ene 28 13:14 snponly.efi
-rwxrwxrwx  1 fogproject root   97740 ene 28 13:14 undionly.kkpxe
-rwxrwxrwx  1 fogproject root   97788 ene 28 13:14 undionly.kpxe
-rwxrwxrwx  1 fogproject root   97741 ene 28 13:14 undionly.pxe
-rwxrwxrwx  1 fogproject root   29728 ene 28 13:14 vesamenu.c32

@george1421
this is my dnsmasq config

And this is the log, i don’t see the target computer 192.168.96.61

root@SRVOPTIFOGADMIN:/var/log# grep -R -i tftp *
auth.log:Jan 28 12:59:51 SRVOPTIFOGADMIN groupadd[3428]: group added to /etc/group: name=tftp, GID=121
auth.log:Jan 28 12:59:51 SRVOPTIFOGADMIN groupadd[3428]: group added to /etc/gshadow: name=tftp
auth.log:Jan 28 12:59:51 SRVOPTIFOGADMIN groupadd[3428]: new group: name=tftp, GID=121
auth.log:Jan 28 12:59:51 SRVOPTIFOGADMIN useradd[3432]: new user: name=tftp, UID=114, GID=121, home=/var
auth.log:Jan 28 12:59:51 SRVOPTIFOGADMIN usermod[3437]: change user 'tftp' password
auth.log:Jan 28 12:59:51 SRVOPTIFOGADMIN chage[3442]: changed password expiry for tftp
auth.log:Jan 28 12:59:51 SRVOPTIFOGADMIN chfn[3445]: changed user 'tftp' information
auth.log:Jan 29 13:25:00 SRVOPTIFOGADMIN polkitd(authority=local): Operator of unix-process:11807:770878 action org.freedesktop.systemd1.manage-units for system-bus-name::1.20 [systemctl restart tftpd-hpa] (o
auth.log:Jan 29 13:27:23 SRVOPTIFOGADMIN polkitd(authority=local): Operator of unix-process:11967:772326 action org.freedesktop.systemd1.manage-units for system-bus-name::1.23 [systemctl restart tftpd-hpa] (o
dpkg.log.1:2020-01-28 12:59:49 install tftpd-hpa:amd64 <ninguna> 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:49 status half-installed tftpd-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:49 status unpacked tftpd-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:49 status unpacked tftpd-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:50 configure tftpd-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1 <ninguna>
dpkg.log.1:2020-01-28 12:59:50 status unpacked tftpd-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:50 status unpacked tftpd-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:50 status unpacked tftpd-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:50 status half-configured tftpd-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:52 status installed tftpd-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:57 install tftp-hpa:amd64 <ninguna> 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:57 status half-installed tftp-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:57 status unpacked tftp-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:57 status unpacked tftp-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:58 configure tftp-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1 <ninguna>
dpkg.log.1:2020-01-28 12:59:58 status unpacked tftp-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:58 status half-configured tftp-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
dpkg.log.1:2020-01-28 12:59:58 status installed tftp-hpa:amd64 5.2+20150808-1ubuntu1.16.04.1
syslog:Feb  1 10:44:38 SRVOPTIFOGADMIN dnsmasq[25135]: opciones al compilar: IPv6 GNU-getopt DBus i18n I
syslog:Feb  1 10:49:26 SRVOPTIFOGADMIN dnsmasq[25529]: opciones al compilar: IPv6 GNU-getopt DBus i18n I
syslog:Feb  1 10:57:26 SRVOPTIFOGADMIN dnsmasq[1130]: opciones al compilar: IPv6 GNU-getopt DBus i18n ID
syslog:Feb  1 10:57:26 SRVOPTIFOGADMIN systemd[1]: Starting LSB: HPA's tftp server...
syslog:Feb  1 10:57:26 SRVOPTIFOGADMIN tftpd-hpa[1274]:  * Starting HPA's tftpd in.tftpd
syslog:Feb  1 10:57:26 SRVOPTIFOGADMIN tftpd-hpa[1274]:    ...done.
syslog:Feb  1 10:57:26 SRVOPTIFOGADMIN systemd[1]: Started LSB: HPA's tftp server.
syslog:Feb  1 12:45:05 SRVOPTIFOGADMIN systemd[1]: Stopping LSB: HPA's tftp server...
syslog:Feb  1 12:45:05 SRVOPTIFOGADMIN tftpd-hpa[8367]:  * Stopping HPA's tftpd in.tftpd
syslog:Feb  1 12:45:05 SRVOPTIFOGADMIN tftpd-hpa[8367]:    ...done.
syslog:Feb  1 12:45:05 SRVOPTIFOGADMIN systemd[1]: Stopped LSB: HPA's tftp server.
syslog:Feb  1 12:45:05 SRVOPTIFOGADMIN systemd[1]: Starting LSB: HPA's tftp server...
syslog:Feb  1 12:45:05 SRVOPTIFOGADMIN tftpd-hpa[8375]:  * Starting HPA's tftpd in.tftpd
syslog:Feb  1 12:45:05 SRVOPTIFOGADMIN tftpd-hpa[8375]:    ...done.
syslog:Feb  1 12:45:05 SRVOPTIFOGADMIN systemd[1]: Started LSB: HPA's tftp server.
syslog:Feb  1 12:48:13 SRVOPTIFOGADMIN systemd[1]: Stopping LSB: HPA's tftp server...
syslog:Feb  1 12:48:13 SRVOPTIFOGADMIN tftpd-hpa[8591]:  * Stopping HPA's tftpd in.tftpd
syslog:Feb  1 12:48:13 SRVOPTIFOGADMIN tftpd-hpa[8591]:    ...done.
syslog:Feb  1 12:48:13 SRVOPTIFOGADMIN systemd[1]: Stopped LSB: HPA's tftp server.
syslog:Feb  1 12:48:13 SRVOPTIFOGADMIN systemd[1]: Starting LSB: HPA's tftp server...
syslog:Feb  1 12:48:13 SRVOPTIFOGADMIN tftpd-hpa[8600]:  * Starting HPA's tftpd in.tftpd
syslog:Feb  1 12:48:13 SRVOPTIFOGADMIN tftpd-hpa[8600]:    ...done.
syslog:Feb  1 12:48:13 SRVOPTIFOGADMIN systemd[1]: Started LSB: HPA's tftp server.

fernando.martinez

@Sebastian-Roth @george1421

FYI,

I have tried downloading “undionly.kpxe” whith this comand “tftp -i 192.168.96.204 get undionly.pxe” on my tftp windows client and it seems that I download it correctly.

Any idea? Thanks

Sebastian Roth

@fernando-martinez Manual TFTP download on a Windows machine is a great test. Should have suggested that way earlier. Now we know TFTP is working.

Re-read the whole topic I noticed that we see VMware, Inc. in one of the pictures. Can you tell us more about your setup? I guess you have VMware Workstation or ESXi server running and try to PXE boot from the FOG server. Can you please try some physical machine within your network as well? Just your normal Windows PC will do, reboot and select PXE boot as temporary first boot. This won’t cause any trouble on the PC. Just to see if this is booting to the menu.

fernando.martinez

@Sebastian-Roth
We are trying on ESXI VM Machine, but we have the same problem whith a physical machine from my network.

We have another Fog Server on another networwk running correctly whith physicals and VM machines.
The only diference is the dhcp server, the first is Palo Alto and the other is Winows Server DHCP.

Sebastian Roth

@fernando-martinez There must be something we overlook here but I can seem to get it. You can manually download the file via Windows tftp command so it should work.

Would you please try to capture a network packet dump on your FOG server when this PXE issue happens so we see what is actually sent over the wire? https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue

george1421

@Sebastian-Roth Collecting the pcap from a witness computer if the target is on a different vlan from the FOG server, or from the FOG server to get the best info if the target computer is on the same vlan as the FOG server.

@fernando-martinez if your target computer is on a different vlan than the fog server install wireshark on a witness computer and use the capture filter of port 67 or port 68

If the target computers is on the same subnet as the fog server then use the instructions in the tutorial.

Upload the pcap to a file share site (i.e. google drive, dropbox, etc) and share as public read. Either post the link here or DM either Sebastian or myself and we will look at the pcap and tell you what we see. Be sure to use the capture filter outlined so we do see things we shouldn’t in your packet capture.

fernando.martinez

@george1421 said in PXE-E32: TFTP open time out on palo alto dhcp server:

uter is on a different vlan than the fog server install

@Sebastian-Roth

this is de output.pcap.
Fog Server: 192.168.96.204
Fog Client: 192.168.96.182

The server and client are in the same Vlan.
Do you need more info?

Thanks

fernando.martinez

@george1421 @Sebastian-Roth

https://www.dropbox.com/s/fn4b256v81ttvlg/output.pcap?dl=0

here is the pcap file.

george1421

@fernando-martinez I’ll be able to look at the pcap file in detail in a few minutes.

The first pass at it I see a problem. The client is asking for undionly.kpxe.0 which is an indication that you have a version of dnsmasq on your fog server that is older than version 2.75. Older versions than that always appended .0 for some legacy reason. You should upgrade to a newer version of dnsmaq, but you can trick it by creating a sym link between undionly.kpxe.0 and undionly.kpxe in the /tftpboot directory. You will need to do the same for ipxe.efi to ipxe.efi.0

fernando.martinez

@fernando-martinez said in PXE-E32: TFTP open time out on palo alto dhcp server:

tftp -i 192.168.96.204 get undionly.pxe

Dnsmasq version is 2.75, I tried to create a sym link but i have the same problem.

george1421

@fernando-martinez would you grab another pcap with these links in place?

What I find strange is that why is it appending the .0 to the file name and why does it appear to request the file and appear to download it. From your picture of the pcap (it appears you deleted the original pcap) the client asks for the file size of undionly.kpxe.0 and it appears to get the file size because it asks for the file next from 96.204. If that file did not exist it should have not attempted to download it but fail on the file size request.

Edit: Ah I had the version wrong it needs to be 2.76 or later ref: https://forums.fogproject.org/topic/12796/installing-dnsmasq-on-your-fog-server

That config file “should” work with 2.75 but we will see the .0 file name appended onto the file name like we see in the pcap.

PXE-E32: TFTP open time out on palo alto dhcp server

260

11.6k

17.1k

154.6k