IPXE Chainloading error in certain type of computer

[***Redbob]

Hi,

I still have the ipxe.efi chainloading tantalizing me… I already posted another issue about it (11240) and it worked when I followed instructions to “connect” to a unmanaged switch over building switch “as a workaround”…

…we have here several types of computer with UEFI boot. Some of them is working properly. That’s its behavior:

on the other side, I have some computers will no work, they gave me this kind of error:

Any ideas? As I see, it’s not properly IPXE issue, because some UEFI are booting from IPXE… Is the second type of BIOS “lazy” to process IPXE Boot request??

[***Redbob]

@Sebastian-Roth it’s funny but… I decided change the client IP (added 172.24.12.49 into DHCP Exclusion Zone now client received 172.24.12.65), so it worked!

Maybe some lockout from FW or Core rules… but it’s working, as I figured out on other computer with some specification. Could you check PCAP of 172.24.12.65? It’s here: new PCAP, just to understand it, because I know, FW and Core rules are not the target of this forum

[Sebastian Roth]

@Redbob Well that’s an interesting one. Looking at the mentioned error page (http://ipxe.org/err/0f0a6095) and the iPXE code I think this particular message can only be caused if the other end is sending a TCP connection reset packet. The other end in your case would either be your FOG server or some kind of gateway/firewall/layer-7-switch in between.

Do you remember if it’s always the very exact same message you get if it fails (“Connection reset”)?

Can you reproduce the issue every time with those particular machines? If yes I would ask you to capture the full traffic of that client and post a PCAP file here. Search the forums for “packet capture” or “PCAP” and you will find instructions. Maybe we can gather more valuable information to figure out what’s going on.

[***Redbob]

@Sebastian-Roth , I did a capture directly from FOG server (172.24.3.144) by Wireshark, the troubled client is 172.24.12.49. Here you have it: https://drive.google.com/open?id=1oolDaPqrF4gFIzN8gRfIXrnyKC18ahHj

[Sebastian Roth]

@Redbob I might have found something in the PCAP already. Though I am not exactly sure what it means as I don’t know you network structure and components. Using thte display filter ip.addr==172.24.12.49 I get to that one single communication of that client very quickly. Now when you look at those packages I see that the client seems to have a MAC addresse ending with ...:89:c2. Very likely this is not the PCs real MAC address but rather the one from the intermediate router/gateway. That’s fine so far. I see always the same MAC throughout the TFTP transfer and the start of the HTTP connection. Now something very weird is happening! Where I would expect a TCP ACK package (to finish up the three way handshake) I see a RST,ACK which naturally just terminates the connection form the servers perspective. Most interestingly this packet is coming from a different MAC address (ends on ...8a:08).

Questions that you need to figure out probably with the help of your network admins:

1. Is ...:89:c2 the correct MAC address of the gateway/router/firewall between the client an the server?
2. Which device has MAC address ...8a:08? Possibly you can figure that out running arp -n on your FOG server of you need to take a look at the ARP tables in your switches.
3. Are there any firewall rules restricting access from this particular client/subnet to the FOG server? More or less unlikely because it would drop/reject the initial TCP SYN packet already.
4. Are you able to access the FOG server web UI from that particular client machine (boot windows and try open (http://172.24.3.144/fog/)?

Possibly this is some kind of magic network intrusion prevention gadget sending those TCP reset packets to the server and client. I am not sure.

[***Redbob]

@Sebastian-Roth these are the answers, as followed:

1. No, MAC address from client is 10:05:01:9c:7b:24 (that’s from 2nd image); MAC adress from FOG Server is 34:64:19:00:26:86;
2. That’s Core Switch (My network is splitted in subnets - FOG is in default subnet - mask 255.255.252.0, the client is in another subnet - mask 255.255.254.0);
3. As I know, there’s no firewall rules restricting LAN-level access, just WAN-level. See IPTABLES from FOG server:

userutil@moringa:~$ sudo iptables -L
[sudo] password for userutil: 
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  anywhere             anywhere             multiport dports mdns
ACCEPT     tcp  --  anywhere             anywhere             multiport dports 4000

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

4. Yes, I could access WEB UI from the client normally.

[Sebastian Roth]

@Redbob As I said, there are two different MAC addresses within the same TCP connection. This is definitely not something normal. One is 40:01:c6:4a:89:c2 which I thought is your router (or layer-7 core switch in your case). And there is another MAC 40:01:c6:4a:8a:08 which is the same vendor but probably too far away from the other one to be some kind of sub-interface MAC in the same device.

Please make sure you find out where those two MACs belong to and let us know. It’s funny, this MAC 40:01:c6:4a:8a:08 is only in 15 packets (7473 in the whole PCAP). One is the TCP reset and the other 14 are ICMP ping requests and replays to/from 172.24.0.1 and 172.24.0.2. Do you have some kind of failover/cluster thing going on there?

4. Yes, I could access WEB UI from the client normally.

Well that’s definitely interesting.

I still have not much of a clue what’s going on here. But my guts tell me that this might be some security feature in the layer-7 switch. Just a wild guess here.

[george1421]

@Sebastian-Roth said in IPXE Chainloading error in certain type of computer:

40:01:c6

FWIW that comes back to 3Com Europe it might just be a IDS/IPS system involved here. A TCP reset is a typical response from a firewall/screening router to a filtered connection requested.

[***Redbob]

@Sebastian-Roth it’s funny but… I decided change the client IP (added 172.24.12.49 into DHCP Exclusion Zone now client received 172.24.12.65), so it worked!

Maybe some lockout from FW or Core rules… but it’s working, as I figured out on other computer with some specification. Could you check PCAP of 172.24.12.65? It’s here: new PCAP, just to understand it, because I know, FW and Core rules are not the target of this forum

[Sebastian Roth]

@Redbob Great stuff we figured this out so quickly. Hope you can dig up what was actually causing the issue for this particular IP address.

Could you check PCAP of 172.24.12.65? It’s here: new PCAP, just to understand it

What exactly are we looking for this time? For this client IP I see the TFTP transfer (default.ipxe) and next is a HTTP POST request (/fog/service/ipxe/boot.php) and a HTTP GET request (/fog/service/ipxe/bg.png) which seems fine from my point of view. The only thing I wonder is one last FIN,PSH,ACK in the TCP session but that is quite common as many clients simply drop the connection without closing it nicely.