Some hosts are unable to get an address through DHCP

[Wayne Workman]

@mageta52 Ok then. This is a switch or hardware issue. What model of computers are you using? and what model of switch? Is it a mini switch or like an enterprise grade Cisco Catalyst ? If it’s a managed switch, what is your configuration? Do you have portfast enabled? Spanning tree? 802.11x power saving options?

[mageta52]

These are custom build 1RU’s with an Asrock Z97E motherboard.

The switch is a D-link DGS 1024D switch. I looked it up to confirm that it is indeed unmanaged

[Wayne Workman]

@mageta52 Can you try a different boot file? It might help. Right now, for legacy, you have undionly.kkpxe configured.

This is in your /etc/dhcp/dhcpd.conf file. They are labeled pretty well in there, it’s the one named “Legacy”.

If your NIC on the motherboard is realtek, use realtek.kpxe and if it’s intel, use intel.kpxe

You might also try out ipxe.kpxe as well.

The computers need fully turned off and back on for the settings to take right. On the fog server, after making a change, restart dhcp with systemctl restart dhcpd

Also, do you know if the motherboard is operating in BIOS mode or UEFI mode? The above instructions are for BIOS.

[Sebastian Roth]

@mageta52 said:

I tried /release /renew a couple of times on the machines that failed to get an address during PXE and they could both get an address when booted into windows. DHCP seems to be working fine.

Sounds very much like a spanning tree issue to me. Can you please try connecting an unmanaged mini switch in between the client and your D-link DGS 1024D switch. Does PXE boot work then? Search the wiki for spanning tree and port fast!

[mageta52]

Looks like they’re running BIOS, not UEFI. The nic is an Intel NIC.

By modifying the boot file I get the same error for both of your suggestions.
“Waiting for link up on net0… Down (http://ipxe.org/38086101)
DHCP failed, hit S for pxe shell, rebooting in 10 seconds”

[mageta52]

@Sebastian-Roth I checked the manual for this switch and the only mention of spanning tree is in the glossary. It gives no mention of the feature anywhere else, and I don’t think that a flat switch like this even supports it.

[Wayne Workman]

@Sebastian-Roth

@mageta52 said in Some hosts are unable to get an address through DHCP:

The switch is a D-link DGS 1024D switch. I looked it up to confirm that it is indeed unmanaged

[mageta52]

@Wayne-Workman So, I took the switch out of there and put in a different one; Trendnet TE100-S16, which is another unmanaged switch.

The pattern I’m seeing is that one host will make it through and get registered, then I move up to the next PC, and it fails to get through DHCP.

I’m not sure what the deciding factor is on whether or not they get through, but I’ve never had more than one get through consecutively

[Sebastian Roth]

@mageta52 Are you familiar with capturing a packet dump (network packets) from the wire using wireshark or tcpdump? This might be really helpful!

[mageta52]

@Sebastian-Roth I could try analyzing the traffic coming into the fog server during an attempted PXE boot to see what’s going on, I’ll report back in a while with the findings.

[Sebastian Roth]

@mageta52 You are more than welcome to upload a pcap file here in the forums or send me a private message if you need help with finding the issue in the packet dump. As you can read in the forums we’ve done this a couple of times and I feel this is one of the best ways to help people debugging their network issues. When you start looking at the packets and understanding what’s going on - this is when you find the solution. No worries, we’ll help you with that.

[mageta52]

Once again, the same pattern, machine 1 gets through and can register, machine 2 fails and stalls out.

I’m not terribly familiar with Tcpdump, but it’s built in, so this is what I got from the second machine…

16:52:11.886022 IP 192.168.235.52.bootps > 255.255.255.255.bootpc: UDP, length 300
16:52:14.893389 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: UDP, length 548
16:52:14.893609 IP 192.168.235.52 > 192.168.235.17: ICMP echo request, id 35325, seq 0, length 28
16:52:15.887222 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:15.894726 IP 192.168.235.52.bootps > 255.255.255.255.bootpc: UDP, length 300
16:52:16.889224 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:17.891224 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:18.902506 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: UDP, length 548
16:52:18.902730 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:19.903230 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:19.903360 IP 192.168.235.52.bootps > 255.255.255.255.bootpc: UDP, length 300
16:52:20.905230 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:22.911623 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: UDP, length 548
16:52:22.911836 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:23.912965 IP 192.168.235.52.bootps > 255.255.255.255.bootpc: UDP, length 300
16:52:23.913229 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:24.915229 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:26.920739 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: UDP, length 548
16:52:26.920963 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:27.922086 IP 192.168.235.52.bootps > 255.255.255.255.bootpc: UDP, length 300
16:52:27.923228 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:28.925225 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:30.929858 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: UDP, length 548
16:52:30.930083 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:31.931204 IP 192.168.235.52.bootps > 255.255.255.255.bootpc: UDP, length 300
16:52:31.931234 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:32.933228 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:34.938973 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: UDP, length 548
16:52:34.939191 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:35.939229 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:35.939363 IP 192.168.235.52.bootps > 255.255.255.255.bootpc: UDP, length 300
16:52:36.941229 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:38.948090 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: UDP, length 548
16:52:38.948310 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:39.949434 IP 192.168.235.52.bootps > 255.255.255.255.bootpc: UDP, length 300
16:52:39.951227 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28
16:52:40.953226 ARP, Request who-has 192.168.235.17 tell 192.168.235.52, length 28

It looks like it wants to assign 192.168.235.17, but is unable to. My guess is that the arp is to make sure that the address is not in use already, but I’m not able to figure out if that’s coming from the PC, or from the fog server? Unfortunately I lost the successful exchange with the first machine. The output got blown away by the data transfer during the machine registration. If more info is needed I can provide it. If there are any switches to turn on with tcpdump for better results let me know and I’ll run the test again.

[Wayne Workman]

@mageta52 Those are just ARP broadcasts. We need to see everything. Look here: https://wiki.fogproject.org/wiki/index.php?title=Troubleshoot_TFTP#Troubleshooting
There’s instructions in there for TCPDump.

[Sebastian Roth]

@mageta52 This is not looking bad. But we need the full “content” of all those packets. You just need to add the -w command line parameter to dump to a file. As well a filter is probably a good idea: tcpdump -w /tmp/dhcp_works_sometimes.pcap udp or arp. Leave the command sitting there and do your client bootups. After success and failure stop tcpdump (ctrl + c) and upload that file to the forum.

[mageta52]

@Sebastian-Roth I’m completely swamped at work this week, and have to abandon this for a while, but I hope to capture the data on Monday.

[Sebastian Roth]

@mageta52 All good. Just get back to us when you have time and I am sure we can get you(r DHCP) sorted…

[mageta52]

@Sebastian-Roth 0_1463176833850_dhcp_fail.pcap

Never uploaded a file before, but there is my attempt, hopefully it can be salvaged.

This was 1) a successful connection from a machine, with full registration, then 2) unsuccessful connection attempt by second machine. Got hung up looking for a DHCP address.

[Sebastian Roth]

@mageta52 Thanks for the packet dump. I see a perfect first boot. Seems all fine as you said. The only thing that got my attention was the time between DHCP discovery request sent by the client and DHCP offer sent back by the server. There is a one second delay which I’ve never seen before I think. Let’s keep that in mind although I am not sure where this is coming from and if that might play a role here.
Then after the first successful boot I see a nice DHCP discovery request send by a different client (MAC address). Again, one second delay followed by a DHCP offer. Although I am not exactly sure it is making it to the client I guess it does (as all the other communication is fine). Then I see a couple more discovery/offer pairs but no request/ack to properly finish the DHCP talk.

Another odd thing I see are a lot of ARP requests from your server. It keeps asking “who is 192.168.235.1”. Either you have this IP configured as DNS server (cat /etc/resolv.conf) or as default gateway (route -n) plus maybe an external DNS server.

Just a wild guess. Your server is trying to resolve a reverse DNS entry before handing out the IP!!! Taking a second for the timeout which then confuses the client…

Ha, wait a second. I think I might have found something else. When the first client requests an IP the server asks “who has 192.168.235.17” to check if this IP is in use already. That’s perfectly fine. But then the next client comes and asks for an IP and I see the server offering the same IP to the different client and again sending an ARP broadcast “who has 192.168.235.17”. Possibly you are running out of leases??? Check the system logs while this is happening: tail -f /var/log/syslog | grep dhcp (or maybe /var/log/messages or /var/log/daemon.log)

Maybe a simple restart of the DHCP service can fix this? Sure you haven’t changed your DHCP config? The one you posted seems fine (range from .235.10 to .235.254).

[mageta52]

@Sebastian-Roth

Alright, so here’s the deal with the arps for 192.168.235.1; That subnet is actually our production network. I installed FOG on that subnet, and then when I want to image I just put it on an isolated, unmanaged switch with the other clients. The gateway is still there though, so it continues to look for it, even when it’s not connected to the production network.

Regarding the DHCP issue, earlier in the week, I allowed the client machines to boot to their old Windows install, and they were able to get addresses just fine. If it was an issue of exhausting the pool, I should have seen it there. On Monday I can try this again to confirm. I can look at the logs as well to see if there is anything.

Will the logs show how many addresses are leased? Is there some place i can check?

[Wayne Workman]

@mageta52 Why can’t you just leave the fog server on the production network? Maybe imaging will happen properly there?