Trouble with Fog and PFSense

[GrumpyRobot]

So I have a fresh install of FOG on Ubuntu 20.04. Been trying to get PXE to work on my homelab with it via PFSense. However, I can’t get anything to boot. I have three VLANs set up, so I put the PXE rules under my wired VLAN which is what the test machines would be on, however, I just get PXE boot failures. Any guidance would be greatly appreciated.

Screenshot of settings in PFSense

[george1421]

@grumpyrobot The only incorrect setting is for the uefi 32 bit. It should be i386-efi/ipxe.efi. But it would be rare that you have a 32 bit efi system.

The PXE boot information goes hand in hand with the dhcp server. So in your case that would mean you need to have the pfsense dhcp server configured on each vlan where you need to pxe boot. There are other ways to go about this if you have a home lab and your isp router is your dhcp server.

[GrumpyRobot]

@george1421 I’ll get that changed. I have my Comcast router in bridge mode to my PFSense box. I have these settings on my wired VLAN which is what my VM and physical machine connect to. Anything else I can look at?

[george1421]

@grumpyrobot well, I think here I might start with wireshark running on a witness computer (3rd computer on same subnet).

Start up wireshark with the capture filter of port 67 or port 68 Then pxe boot the target computer. Keep recording until you get the error on the target computer.

Now look at your wireshark screen. You should see a standard DORA sequence.

First you will see a DISCOVER from the target computer.
The second packet (OFFER) will be from your dhcp server (should be your pfsense server). You will want to inspect this packet. In the ethernet header there are 2 fields of interest. {next server ip address} and {boot file name}. The next server field should have the IP address of the FOG server. The boot file name should be undionly,kpxe for bios or ipxe.efi for uefi.

If those are set then the bootp part is set correctly. Next scroll down until you see the dhcp options then expand dhcp option 66 and 67. Those values should match the ethernet header values.

My bet something here is wrong.

[GrumpyRobot]

@george1421 I tried running Wireshark and don’t see option 66 or 67 listed in my Discover request. Maybe I’m looking at the wrong portion (not super familiar with Wireshark). Here are the images I took of my boot screen (which shows my FOG ip) and of Wireshark (which doesn’t). Hopefully this helps.

WireShark

Computer booting

[george1421]

@grumpyrobot You want the OFFER not DISCOVER. DISCOVER comes from the pxe booting computer, OFFER comes from dhcp server.

So from the screen shot is 192.168.1.49 your FOG sever?

If so we need to look into why either the tftp server isn’t running or you have a the linux firewall enabled on the FOG server. What you are looking using wireshark is displayed in the screen shot. Not all pxe boot roms are that helpful.

[GrumpyRobot]

@george1421 I’m going to guess I am doing something wrong on my end. I do not see OFFER on Wireshark, only DISCOVER and REQUEST. Should I be running Wireshark on the FOG computer, or on another computer in my network?

[george1421]

@grumpyrobot Well dhcp uses broadcast messages to communicate, every once and a while you will find that a dhcp server will use unicast messages for replying. This is rare but it happens. Unicast messaging can’t be detected with a witness computer unless you have port mirroring enabled.

But from what we see in the bios screen, I want to focus on the fog server, specifically two points.

1. Is 192.168.1.49 your fog server IP address? If so can you connect to it using putty or ssh from another computer on the 192.168.1.x subnet?
2. Can you reach the tftp server from a computer on the 192.168.1.x subnet. If you have a windows 10 computer you can add the tftp client feature. Once installed you will make sure you disable the windows firewall (for testing) and then use the tftp client from a command window to see if you can download the ipxe boot loader with the command tftp -i 192.168.1.49 GET undionly.kpxe . I’m only interested to see if the download works.

The bios screen error indicates either the pxe booting computer can’t reach the FOG server or the tftp service is not passing the boot loader to the pxe booting computer.

[GrumpyRobot]

@george1421 I can SSH into it just fine from other machines. The TFTP, however, gives me the “Connect Request Failed” error message. Is there a port or something I need to enable on my Linux machine for TFTP on Ubuntu 20.04?

[GrumpyRobot]

@george1421 Also it looks like TFTP is running, just not reachable

[george1421]

@grumpyrobot ok since you have a ubuntu system key in this command sudo ufw status confirm that the firewall is disabled.

If its enabled use this command sudo ufw disable to turn it off. Then try to tftp again.

[GrumpyRobot]

@george1421 Status shows inactive

[george1421]

@grumpyrobot said in Trouble with Fog and PFSense:

@george1421 Status shows inactive

Well this should not be then.

From the 192.168.1.x subnet you can ping the fog server, you can connect to the web ui, you can ssh to it. The tftp service is running. The ubuntu firewall is disabled.

In short it has to be working, can the fog server connect to its self using the tftp program and command line?

[GrumpyRobot]

@george1421 I think I found the root of the issue. I installed it originally with https, but it looks like the cert was not validated. I resintalled without https and it seems to work just fine now. Is there a way to validate the cert with https? The wiki mentions system time can cause an issue, but that was all correct, so my guess is something else may have broke the cert.