PXE-E99: Unexpected Network Error

[alexnoel2]

Good morning,

After a lot of trial and tribulation myself and a network engineer were able to get our Fog (Ubuntu 18.04) server to deploy to the location we need it to. Bad news, it is only deploying to legacy machines. The laptops we need to image are Panasonic CF-54 Toughbooks, and they are all UEFI. Secure boot is disabled. We finally managed to snag the pop up message and it read as follows.

Station IP address is 172.25.25.60
Server IP Address is 172.25.25.50
NBP filename is ipxe.efi
NBP filesize is 0 bytes
PXE-E99: Unexpected Network Error.

If anybody has a clue, please let me know because I am lost. Unfortunately, I cannot provide screenshots of the activity or provide logs.

[george1421]

@alexnoel2 Since the FOG server and target computer are on the same subnet I would use tcpdump on the fog server to see what communication is going on between the dhcp, client, and fog server. https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue

You can review the captured pcap with wireshark. Since you are not able to provide the file you will need to look through it to see what is failing. You should see the DISCOVER, OFFER, REQUEST, ACK process of dhcp and right after that you should see the target computer connect to the FOG server first to confirm the file size and second to download the file. I suspect this is where its failing.

Just out of curiosity, what device is your dhcp server (mfg and model)?

[alexnoel2]

@george1421 The FOG Server is the DHCP server, it is an Ubuntu 18.04.5 VM hosted on ESXi 6.5, which is hosted on a Dell R340. 2 Interfaces across one physical NIC, one to the corporate network, and the other for PXE boot.

[george1421]

@alexnoel2 And when you installed FOG you select the right interface to bind the dhcp server too? I assume from your post that FOG was working fine on bios but uefi not so good?

If you are using the FOG server for dhcp services, then it should automatically support both bios and uefi computers. There should be nothing you need to do to make it pxe boot into the iPXE menu.

EDIT: sorry i missed you have it virtualized and the fog server might not have 2 interfaces but the host server does.

[alexnoel2]

@george1421 The Fog server does have 2 interfaces, they are just using the same physical nic. So I looked through the pcap in Wireshark and came across the following.

Option (129-135)
Parameter Request List Item: PXE - undefined (Vendor Specific)
Option (60)
Vendor class identifier: PXEClient:Arch:00007:UNDI:003016

Does that ring any bells? Other than that I didnt find anything glaring. If you know of something specific to look for it would be helpful.

[george1421]

@alexnoel2 Well it could be a few things and nothing its hard to explain because there is a lot of exceptions. You want to look in the OFFER in both the ethernet header there is a next-server and boot-file that should point to the ip address of fog server and ipxe.efi file. Also dhcp option 66 and 67 should be the same.

Do you see the tftp query from the target computer to the FOG server asking for the ipxe.efi file?

You said something earlier that the network engineer did something to make this work. What did the engineer touch? Because this should have just worked out of the box.

Just to be clear if you put a bios computer on this network it will pxe boot into the iPXE menu just fine? The only thing that is changing is bios to uefi? On the bios computer does it also support uefi mode? So on the same computer if you change it from bios to uefi mode does it pxe boot into the iPXE menu?

Understand I’m coming cold into the problem not knowing your environment or what might have been changed. So I have to give a lot of guesses until we narrow in on where the problem is.

[alexnoel2]

@george1421 Just to clarify, the server is specifically not deploying to these laptops. We were able to deploy to laptops with legacy bios from the same location.

[george1421]

@alexnoel2 said in PXE-E99: Unexpected Network Error:

We were able to deploy to laptops with legacy bios from the same location

So if you take (for a test) switch one of these laptops from bios to uefi mode (on the same network jack) can you get into the iPXE menu? We don’t need to image them only get into the iPXE menu.

[alexnoel2]

@george1421 So far, no.

[george1421]

@alexnoel2 Ok so let me ask again because I’m not absolute about your answer.

1. You can take the same computer and in bios mode does it boot into the iPXE menu?

2. If you switch that same computer into uefi mode does it boot into the iPXE menu?

We are trying to find the exact point of failure. If the answer is yes in bios mode and no in uefi mode then lets focus on uefi boot file.

For the uefi boot file, if you have a computer on the same network install the tftp client in windows, you will need to temporary drop the windows firewall on the test computer, but then use tftp to get ipxe.efi from the fog server. We are only concerned that it download the file. We don’t need it only to download the file.

If the tftp works then we need to find out exactly what the network admin did to make this work.

[alexnoel2]

@george1421 1. Yes. 2. No.
I only have an issue in UEFI. The laptops I need to image, do not have a legacy option. Why is the laptop, not able to pull from ipxe.efi? I am sorry, I just feel like we are on 2 different pages right now.

[george1421]

@alexnoel2 I don’t think we are too far apart. So let me explain.

If you are using the same network connection, same computer and the only thing you are changing is putting the computer into one more or another, then we’ve ruled out a bunch of possibilities. Its either the computer, the dhcp server configuration, or ipxe.efi is missing on the fog server (for some reason).

I understand these computers in question only have uefi mode. So do you have a different model that supports both modes for testing? We need to test everything the same except the computer mode bios/uefi because you said bios mode works perfectly.

[Sebastian Roth]

@alexnoel2 said in PXE-E99: Unexpected Network Error:

NBP filesize is 0 bytes

Not sure if I get this right but this might point to the file ipxe.efi not being available on the TFTP server? What do you get when running ls -al /tftpboot/ipxe.efi?

If you are missing that file you can quickly download it here: https://github.com/FOGProject/fogproject/tree/master/packages/tftp

[alexnoel2]

@george1421 @Sebastian-Roth Just wanted to give you an update. From what I can tell, the section of dhcpd.conf where you declare options, was missing a } so the file wasn’t reading the options I guess? I compared the file to an old server and that was the only difference I could find. I probably deleted it while editing the file at some point. Works like a charm now.