"Please enter tftp server:" with single dhcp server

[xcess]

I know this might be annoying because I’m not the first one with the problem. I’ve read the other topics but unfortunately still can’t resolve my issue.
The setup is:

• FOG server running on Fedora
• dhcp server running on ExtremeNetworks core-switch.
• everything is running in a VLAN dedicated for testing purposes. No other DHCP servers.
• switch supports dhcp options codes, so I’ve set the following:
  • DHCP Option Code 66 : string “192.168.xxx.xx”
  • DHCP Option Code 67 : string “tftp://192.168.xxx.xx/undionly.kpxe”

Client boots into “Please enter tftp server:”
Providing the IP works.
I’ve investigated what’s going on with Wireshark and can confirm that there’s only one DHCP server messing around.
Here’s a snipped of a bootp frame:

Frame 106: 369 bytes on wire (2952 bits), 369 bytes captured (2952 bits) on interface \Device\NPF_{4C63F991-A548-48B3-A772-5278B943AE7F}, id 0
Ethernet II, Src: ExtremeN_9b:8e:d0 (00:04:96:9b:8e:d0), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 192.168.230.253, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 67, Dst Port: 68
Dynamic Host Configuration Protocol (Offer)
    Message type: Boot Reply (2)
    Hardware type: Ethernet (0x01)
    Hardware address length: 6
    Hops: 0
    Transaction ID: 0x811fa173
    Seconds elapsed: 0
    Bootp flags: 0x8000, Broadcast flag (Broadcast)
    Client IP address: 0.0.0.0
    Your (client) IP address: 192.168.230.200
    Next server IP address: 0.0.0.0
    Relay agent IP address: 0.0.0.0
    Client MAC address: 82:5d:c6:8a:7e:48 (82:5d:c6:8a:7e:48)
    Client hardware address padding: 00000000000000000000
    Server host name not given
    Boot file name not given
    Magic cookie: DHCP
    Option: (51) IP Address Lease Time
    Option: (53) DHCP Message Type (Offer)
    Option: (1) Subnet Mask (255.255.255.0)
    Option: (54) DHCP Server Identifier (192.168.230.253)
    Option: (3) Router
    Option: (6) Domain Name Server
    Option: (66) TFTP Server Name
        Length: 14
        TFTP Server Name: 192.168.xxx.xx
    Option: (67) Bootfile name
        Length: 35
        Bootfile name: tftp://192.168.xxx.xx/undionly.kpxe
    Option: (255) End

Aaand it seems that I’m stuck. Any help will be appreciated.

[george1421]

@xcess said in "Please enter tftp server:" with single dhcp server:

DHCP Option Code 67 : string “tftp://192.168.xxx.xx/undionly.kpxe”

^^ This value is wrong ^^, just enter undionly.kpxe for bios based computers and ipxe.efi for uefi based computers.

[george1421]

@xcess said in "Please enter tftp server:" with single dhcp server:

Server host name not given
Boot file name not given

This will be a problem too. That should align up with dhcp options 66 and 67.

So now to the question, what is your dhcp server (mfg/model)?

[xcess]

@george1421 said in "Please enter tftp server:" with single dhcp server:

@xcess said in "Please enter tftp server:" with single dhcp server:

DHCP Option Code 67 : string “tftp://192.168.xxx.xx/undionly.kpxe”

^^ This value is wrong ^^, just enter undionly.kpxe for bios based computers and ipxe.efi for uefi based computers.

Greetings, George. Initially I tried with what you’re suggesting but it didn’t work. Then the maintainer of iPXE suggested using the whole address with tftp url and that provided result.

@george1421 said in "Please enter tftp server:" with single dhcp server:

@xcess said in "Please enter tftp server:" with single dhcp server:

Server host name not given
Boot file name not given

This will be a problem too. That should align up with dhcp options 66 and 67.

So now to the question, what is your dhcp server (mfg/model)?

I’m using the built-in dhcp server on a ExtremeNetworks switch.

[george1421]

@xcess said in "Please enter tftp server:" with single dhcp server:

I’m using the built-in dhcp server on a ExtremeNetworks switch.

OK knowing this are your pxe booting computers on the same subnet as the FOG server? (while I need to know this…)

I would recommend you move away from the network switch providing the PXE boot information for your network. Can you probably make it work, maybe yes. Is there an easier way to go about it and make the pxe booting a bit more dynamic, yes. So the short answer is to install dnsmasq on your FOG server to provide pxe boot info only. The reset of the dhcp stuff will come from your main dhcp server.

If you want to use your switch dhcp sever only, then you must pick either bios or uefi system booting, because the dhcp options are static, so you have to pick one or the other. The problem why it didn’t work the first time is that because the {next-server} and {boot-file} entries were missing in the ethernet header. This is the bootp part of pxe booting where dhcp uses dhcp options 66 and 67. So in your network switch you need to turn on bootp as well as dhcp. Sometimes they are separate tasks and sometimes they are combined. Some clients will look at the ethernet header for boot info and others will use the dhcp options, that’s why both need to be filled out, or just use dnsmasq.

I have instructions on installing dnsmasq, it should take you about 10 minutes start to finish. If you have the pxe booting computers on a different subnet than the fog server, you will need to update you dhcp-relay service to include the fog server ip address.

[xcess]

@george1421

I really wanted to utilize our switch (if possible). As I have already tested this with isc-dhcp server and I know it works.
So far I’ve option codes 66 and 67 on the switch. Also enabled BOOTPRELAY to the FOG server, but still no joy. Do you believe setting up dnsmasq could help me, or should I just stick to the isc-dhcp server?

edit: I have also tried setting option 67 to “http://192.168.230.51/fog/service/ipxe/boot.php” and that works, but loads the FOG menu without the fancy graphics

[george1421]

@xcess said in "Please enter tftp server:" with single dhcp server:

You are a bit all over the place with this :

I really wanted to utilize our switch (if possible).

My suggestion is suggestion is to use your switch as dhcp server and dnsmasq to provide pxe boot information (only)

As I have already tested this with isc-dhcp server and I know it works.

In this case you are replacing your switch dhcp server with isc-dhcp server, not what you wanted.

So far I’ve option codes 66 and 67 on the switch.

As I mentioned before these are for dhcp booting, what is missing from your DHCP OFFER packet is the bootp info in the ethernet header. Both are required because clients can use either.

Also enabled BOOTPRELAY to the FOG server, but still no joy.

You need that for dnsmasq, Good you did this but will not have an impact until you install dnsmasq configured as ProxyDHCP

Do you believe setting up dnsmasq could help me

Yes

https://forums.fogproject.org/topic/12796/installing-dnsmasq-on-your-fog-server

[xcess]

@george1421

Hey George. Thank you so much for taking the time to assist me!
I’ve configured dnsmasq as per your guide, but unfortunately it makes no difference. Bootprelay is enabled and pointing at the dnsmasq server (which is the same machine as the FOG server). My client is receiving IP address from the pool, but afterwards I’m getting “Nothing to boot” message and can’t proceed. My dnsmasq seems to be correct.

[george1421]

@xcess Ok so you have dnsmasq setup. Do you have a way to pxe boot a computer that is on the same subnet as the FOG server? This can be a physical machine or a vm configured to pxe boot. Our goal here is to get to the FOG iPXE menu. If you have dnsmaq setup and running on your fog server the target computer should pxe boot.

This test will tell us if its the FOG server (dnsmasq or tftp) that is at fault or your network infrastructure.

If it pxe boots then we go to the next step of installing wireshark on a witness computer (third computer plugged into same subnet as your remote computer who needs to pxe boot). Use a wireshark capture filter of port 66 or port 67 then pxe boot the target computer to the error. Look at the packet capture. You should (must) see 2 DHCP OFFER packets. One from your main dhcp server and one from dnsmasq running on the FOG server. I suspect this second OFFER is missing.

[xcess]

@george1421

Yes, my test client is on the same subnet (192.168.230.0/24). It is a virtual machine. I also have physical computer in the same subnet, that I’m using to examine frames with Wireshark.

My dnsmasq is running on the fog server, but my test virtual machine won’t pxe boot. In Wireshark there’s only two discover packets and one REQUEST coming from the VM. Seems like my FOG machine isn’t involved at all! Maybe after all it could be issue with the switch.

[george1421]

@xcess said in "Please enter tftp server:" with single dhcp server:

I also have physical computer in the same subnet, that I’m using to examine frames with Wireshark.

Ok this will make debugging a bit easier when everyone on the same subnet. Lets use tcpdump on the fog server (this is the best place to grab the packets because the fog server will listen to both broadcast and unicast messages sent to the fog server) https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue Also inspect the pcap in wireshark. Again looking for the 2 offers. At the very least you should see the offer from dnsmasq.

[george1421]

@george1421 Now if you are seeing only the OFFER packet from your dhcp server but no OFFER from dnsmasq (and possibly no DISCOVER packet from your target computer) then it sounds like you have the firewall enabled on the FOG server. You will need to make exceptions for bootp/dhcp as well as tftp to pxe boot. Its also possible that dnsmasq is not running for some reason (like a bad configuration) you can view if dnsmasq is running by keying in sudo ps aux|grep dnsmasq and netstat -an | grep :67

If you don’t see any DISCOVER or OFFER look at your FOG Server firewall.

[xcess]

@george1421

I’ve stopped and disabled the firewall since I configured the FOG machine. It’s inactive.

dnsmasq seems to be running:

ps aux|grep dnsmasq
dnsmasq    66328  0.0  0.0  25232  3912 ?        Ss   15:10   0:00 /usr/sbin/dnsmasq -k

netstat -an | grep :67
udp        0      0 0.0.0.0:67              0.0.0.0:*

It seems to me that the switch won’t relay anything to the FOG machine. I’ve also tested all this with and without options 66 & 67 set on the dhcp server.

[george1421]

@xcess OK so we know (from your screen shots) that dnsmasq is up and running. I assume your tcpdump did not show a DISCOVER or OFFER packet, so I agree with your assessment something in your infrastructure is blocking these broadcast messages. If dnsmasq doesn’t hear the DISCOVER packet it will not give up an OFFER packet.

If your infrastructure is blocking dhcp broadcast messages then your witness computer should also not see any external DISCOVER, OFFER, REQUEST, ACK, or INFORM dhcp messages too.

For some reason dhcp-snooping comes to mind. That might limit the broadcast messages related to the DHCP DORA process to only the computers involved with DHCP startup. It really depends on the switch mfg.

[george1421]

Some quick google-fu finds this information

https://community.extremenetworks.com/extremeswitching-exos-233219/acl-to-block-dhcp-bootp-pxeboot-from-all-but-certain-servers-6965720?postid=16750946#post16750946

Where dhcp-snooping might be causing the issue with dnsmasq since the FOG server may not be on a trusted port.

And this one
https://community.extremenetworks.com/faqs-233202/configuring-dhcp-on-the-securestacks-5839969

Specifically these commands

enable Enable automatic address allocation for bootp clients.
disable 
Disable automatic address allocation for bootp
clients. [default]

C2(rw)->set dhcp enable ?
C2(rw)->set dhcp bootp enable ?
C2(rw)->set dhcp bootp disable ?

Where specifically the dhcp OFFER packet from your switch is missing the ethernet header part of the options, maybe the bootp bits of your dhcp configuration is not turned on

[xcess]

@george1421

Thanks, George. I had a look at this. The first link corresponds to the appliance we have. I had a shot with the mentioned configurations (by adding trusted server and dhcp snooping) but that didn’t change a thing.

The second link relates to securestacks and that doesn’t apply to our switch (EXOS). No such commands here.

At this point it seems easier just to stop switch’s built-in DHCP, re-enable the isc-dhcp server on my FOG machine and point bootprelay to it. That’s my working scenario that I used few days ago, and also what makes everything so frustrating.

[george1421]

@xcess Well that first link looks and gives guidance to you need to enable the fog server network port as trusted so the dhcp communication is echoed to that device.

The second link, I hoped that would turn on what is missing in the initial OFFER packet from your switch dhcp server (two different paths to solving two different problems, to get your clients pxe booting).

    Client MAC address: 82:5d:c6:8a:7e:48 (82:5d:c6:8a:7e:48)
    Client hardware address padding: 00000000000000000000
>>    Server host name not given
>>    Boot file name not given
    Magic cookie: DHCP
    Option: (51) IP Address Lease Time

Now you said that isc-dhcp server worked?? The reason why I ask is that isc-dhcp server and dnsmasq do the same thing. So why would isc-dhcp server issue IP addresses and dnsmasq does not respond?

[xcess]

@george1421

Now you said that isc-dhcp server worked?? The reason why I ask is that isc-dhcp server and dnsmasq do the same thing. So why would isc-dhcp server issue IP addresses and dnsmasq does not respond?

To be honest, this is beyond me.

Initially, when I configured the FOG server I installed isc-dhcp server, configured pool of addresses and added “next-server” and “filename”. Then I set bootprelay of the switch to point to the FOG server and everything worked (and still works if I go back to this scenario).
Afterwards I realized that our expensive switch supports dhcp and decided to give it a shot (why use separate service for something that already runs on the switch, right?). Since then I’ve encountered this issue i.e. switch supporting only numerical option codes (66, 67 etc). Analyzing the frames turned out that my clients are unable to evaluate those options even though they do exist in the dhcp packets! Having “next-server” and “Boot file name” in the packet body seemed like the only way it works. I managed to confirm this by analyzing frames coming from my working isc-dhcp server.

Now, you can probably see how strange it is.