Intermittent PXE-E32: TFTP Open Timeout
-
I have a confusing issue.
Classroom with 24 workstations at my local school.Here’s my setup:
Two Cisco Catalyst 3560G switches.- SW1 - gateway switch. VLAN 128 - Workstations.
- SW2 - VLAN 50 - servers.
Server3 - Windows Server 2016 with Hyper V running two VMs:
- DC1 - 10.80.50.8 - Windows Server 2016 Running AD, DNS, DHCP
- FOG - Running FOG 1.59 on latest release of Debian.
Server1 - Windows Server 2016 with Hyper V running one VM:
- DC2 - 10.80.50.6 - Windows Server 2016 Running AD, DNS, DHCP
Most recent change was I added in SW2, which is connected to SW1 via Etherchannel (two gigabit connections). Switches are both running STP.
It is not Etherchannel because I had this issue prior to setting up Etherchannel - when the switches were connected via single gigabit uplink.
Both switches have VLAN 50 and 128.The issue is that sometimes I boot two test workstations, they will ALWAYS get a DHCP IP from either DHCP server, but TFTP times out. Other times, TFTP works and it boots to the FOG menu. It’s very intermittent.
The thing that confuses me, is that when DHCP hands off an IP to a PXE client, it shows an additional gateway of 10.80.128.2 - which is the IP address of SW2 VLAN 128. None of my DHCP settings on my servers have this IP setup as a gateway.
I have tried unauthorizing one DHCP server at a time and it didn’t seem to make a difference.
Options 66 and 67 are configured and have been working perfectly, 100% of the time up until adding the second switch.
Any thoughts on what I can try?
-
@tc10284 said in Intermittent PXE-E32: TFTP Open Timeout:
The thing that confuses me, is that when DHCP hands off an IP to a PXE client, it shows an additional gateway of 10.80.128.2 - which is the IP address of SW2 VLAN 128.
What IP do clients get then you see the 10.80.128.2 gateway address on screen? I could imagine they get 10.80.128.x addresses - anything else would not make much sense really. Probably the new SW2 has a DHCP service running causing this issue.
Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)
Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG
-
@sebastian-roth
Check out these two pics. You will see where it shows two gateways for some reason.
https://photos.app.goo.gl/SKANVs1URjYxYvY78As for SW2, it was started with a blank, fresh config and there’s no DHCP server running. I have the appropriate IP helper-address commands setup for the two DHCP/DNS servers.
I’ll also add that I don’t have any issues with DNS or DHCP that I can find.
SW1:
interface Vlan50
description SERVER VLAN
ip address 10.80.50.1 255.255.255.0
!
interface Vlan128
description WORKSTATION VLAN
ip address 10.80.128.1 255.255.255.0
ip helper-address 10.80.50.6
ip helper-address 10.80.50.8SW2:
interface Vlan50
ip address 10.80.50.2 255.255.255.0
!
interface Vlan128
ip address 10.80.128.2 255.255.255.0
ip helper-address 10.80.50.6
ip helper-address 10.80.50.8 -
@sebastian-roth Ah I see it.
The OP needs to set the second switch up as an L2 switch and not an L3 switch (with router). Do this for both vlans on that switch. Make sure the links between the switches are setup as vlan trunks where the links are members of the vlans you want to cross the switch to switch link.
You in a way have one vlan with two routers on it. Because the dhcp server is not on the same subnet as the target computers you have the dhcp helper service running on both routers. They are now duplicating the dhcp requests to the dhcp servers (because both are hearing the dhcp discover packet).
-
Here is DC1 DHCP config:
Here is DC2 DHCP config:
Both are setup in a DHCP failover config, so the config is (and should stay) the same on both.
-
@george1421 said in Intermittent PXE-E32: TFTP Open Timeout:
The OP needs to set the second switch up as an L2 switch and not an L3 switch (with router).
Yes, having two gateways/router in one subnet is not “ideal”…
-
@george1421 said in Intermittent PXE-E32: TFTP Open Timeout:
@sebastian-roth Ah I see it.
The OP needs to set the second switch up as an L2 switch and not an L3 switch (with router). Do this for both vlans on that switch. Make sure the links between the switches are setup as vlan trunks where the links are members of the vlans you want to cross the switch to switch link.
You in a way have one vlan with two routers on it. Because the dhcp server is not on the same subnet as the target computers you have the dhcp helper service running on both routers. They are now duplicating the dhcp requests to the dhcp servers (because both are hearing the dhcp discover packet).
And that’s the other confusing part. I’m no Cisco expert, but I don’t see where I have any routing enabled on SW2. So I’m not sure why PXE or TFTP is picking up SW2 as a gateway also, when it’s not conifguredin DHCP anywhere.
Here are my switch configs for review and if someone could help point that out.
SW1
version 12.2 no service pad no service timestamps debug uptime service timestamps log datetime service password-encryption service sequence-numbers ! hostname SW1 ! boot-start-marker boot-end-marker ! (removed enable passwords) ! ! ! aaa new-model ! ! aaa authentication login default local none aaa authentication login ssh local ! ! ! aaa session-id common clock timezone UTC -5 system mtu routing 1500 ip routing no ip domain-lookup ip domain-name (removed domain name) ip name-server 10.80.50.6 ip name-server 10.80.50.8 ip name-server 8.8.8.8 ! ! ! ! (removed Crypto config) ! ! ! spanning-tree mode pvst spanning-tree portfast default spanning-tree extend system-id ! vlan internal allocation policy ascending ! ip ssh time-out 90 ip ssh authentication-retries 2 ip ssh version 2 ! ! interface Port-channel5 switchport trunk encapsulation dot1q switchport trunk allowed vlan 50,128 switchport mode trunk ! interface GigabitEthernet0/1 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/2 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/3 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/4 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/5 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/6 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/7 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/8 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/9 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/10 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/11 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/12 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/13 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/14 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/15 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/16 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/17 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/18 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/19 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/20 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/21 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/22 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/23 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/24 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/25 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/26 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/27 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/28 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/29 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree portfast spanning-tree bpduguard enable ! interface GigabitEthernet0/30 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/31 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/32 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/33 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/34 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/35 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/36 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/37 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/38 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/39 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/40 switchport access vlan 128 switchport mode access switchport nonegotiate no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/41 switchport access vlan 128 switchport mode access no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/42 switchport access vlan 128 switchport mode access no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/43 switchport access vlan 128 switchport mode access no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/44 switchport access vlan 128 no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/45 switchport access vlan 128 switchport mode access no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/46 switchport access vlan 128 switchport mode access no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/47 switchport access vlan 128 switchport mode access no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/48 switchport access vlan 128 switchport mode access no keepalive storm-control broadcast level 50.00 spanning-tree bpduguard enable ! interface GigabitEthernet0/49 no switchport ip address 10.80.75.2 255.255.255.0 ! interface GigabitEthernet0/50 description SERVER VLAN switchport trunk encapsulation dot1q switchport trunk native vlan 224 switchport trunk allowed vlan 99,224 switchport mode trunk ! interface GigabitEthernet0/51 switchport trunk encapsulation dot1q switchport trunk allowed vlan 50,128 switchport mode trunk channel-group 5 mode desirable ! interface GigabitEthernet0/52 switchport trunk encapsulation dot1q switchport trunk allowed vlan 50,128 switchport mode trunk channel-group 5 mode desirable ! interface Vlan1 ip address 10.80.1.1 255.255.255.0 ip access-group 110 in ! interface Vlan50 description SERVER VLAN ip address 10.80.50.1 255.255.255.0 ! interface Vlan128 description WORKSTATION VLAN ip address 10.80.128.1 255.255.255.0 ip helper-address 10.80.50.6 ip helper-address 10.80.50.8 ! ! router eigrp 3 network 10.80.1.0 0.0.0.255 network 10.80.50.0 0.0.0.255 network 10.80.75.0 0.0.0.255 network 10.80.128.0 0.0.0.255 passive-interface Vlan50 passive-interface Vlan128 eigrp stub connected summary ! ip classless ip route 0.0.0.0 0.0.0.0 10.80.75.1 no ip http server ip http secure-server ! kron occurrence tftpbackup at 18:00 Mon recurring policy-list tftpbackup ! access-list 115 permit ip 10.80.50.0 0.0.0.255 any access-list 115 remark -=[Restrict SSH access]=- no cdp run vstack alias configure dsr do show run alias exec sr show run ! line con 0 exec-timeout 90 0 password (removed) logging synchronous line vty 0 4 access-class 115 in exec-timeout 90 0 password (removed) logging synchronous length 0 transport input telnet line vty 5 15 exec-timeout 0 0 password (removed) no exec transport input telnet ! ntp clock-period 36029217 ntp server 193.136.152.72 endSW2
version 12.2 no service pad service timestamps debug datetime msec service timestamps log datetime msec service password-encryption ! hostname SW2 ! boot-start-marker boot-end-marker ! enable secret 5 $1$fBsL$ob49EQzhePJwm8UzpG1CU0 ! ! ! no aaa new-model system mtu routing 1500 ip domain-name (removed) ip name-server 10.80.50.6 ip name-server 10.80.50.8 ip name-server 8.8.8.8 ! ! ! ! ! ! ! ! spanning-tree mode pvst spanning-tree extend system-id ! vlan internal allocation policy ascending ! ip ssh version 2 ! ! interface Port-channel5 switchport trunk encapsulation dot1q switchport trunk allowed vlan 50,128 switchport mode trunk ! interface GigabitEthernet0/1 switchport access vlan 50 ! interface GigabitEthernet0/2 switchport access vlan 50 ! interface GigabitEthernet0/3 switchport access vlan 50 ! interface GigabitEthernet0/4 switchport access vlan 50 ! interface GigabitEthernet0/5 switchport access vlan 50 ! interface GigabitEthernet0/6 switchport access vlan 50 ! interface GigabitEthernet0/7 switchport access vlan 50 ! interface GigabitEthernet0/8 switchport access vlan 50 ! interface GigabitEthernet0/9 switchport access vlan 50 ! interface GigabitEthernet0/10 switchport access vlan 50 ! interface GigabitEthernet0/11 switchport access vlan 50 ! interface GigabitEthernet0/12 switchport access vlan 50 ! interface GigabitEthernet0/13 switchport access vlan 50 ! interface GigabitEthernet0/14 switchport access vlan 50 ! interface GigabitEthernet0/15 switchport access vlan 50 ! interface GigabitEthernet0/16 switchport access vlan 50 ! interface GigabitEthernet0/17 switchport access vlan 50 ! interface GigabitEthernet0/18 switchport access vlan 50 ! interface GigabitEthernet0/19 switchport access vlan 50 ! interface GigabitEthernet0/20 switchport access vlan 50 ! interface GigabitEthernet0/21 switchport access vlan 50 ! interface GigabitEthernet0/22 switchport access vlan 50 ! interface GigabitEthernet0/23 switchport access vlan 50 ! interface GigabitEthernet0/24 switchport access vlan 50 ! interface GigabitEthernet0/25 switchport access vlan 50 ! interface GigabitEthernet0/26 switchport access vlan 50 ! interface GigabitEthernet0/27 switchport access vlan 50 ! interface GigabitEthernet0/28 switchport access vlan 50 ! interface GigabitEthernet0/29 switchport access vlan 50 ! interface GigabitEthernet0/30 switchport access vlan 50 ! interface GigabitEthernet0/31 switchport access vlan 50 ! interface GigabitEthernet0/32 switchport access vlan 50 ! interface GigabitEthernet0/33 switchport access vlan 50 ! interface GigabitEthernet0/34 switchport access vlan 50 ! interface GigabitEthernet0/35 switchport access vlan 50 ! interface GigabitEthernet0/36 switchport access vlan 50 ! interface GigabitEthernet0/37 switchport access vlan 50 ! interface GigabitEthernet0/38 switchport access vlan 50 ! interface GigabitEthernet0/39 switchport access vlan 50 ! interface GigabitEthernet0/40 switchport access vlan 50 ! interface GigabitEthernet0/41 switchport access vlan 50 ! interface GigabitEthernet0/42 switchport access vlan 50 ! interface GigabitEthernet0/43 switchport access vlan 50 ! interface GigabitEthernet0/44 switchport access vlan 50 ! interface GigabitEthernet0/45 switchport access vlan 50 ! interface GigabitEthernet0/46 switchport access vlan 50 switchport mode access ! interface GigabitEthernet0/47 switchport access vlan 50 ! interface GigabitEthernet0/48 switchport access vlan 50 switchport mode access ! interface GigabitEthernet0/49 switchport access vlan 50 ! interface GigabitEthernet0/50 switchport access vlan 50 switchport trunk encapsulation dot1q switchport trunk allowed vlan 50,128 switchport mode trunk ! interface GigabitEthernet0/51 switchport trunk encapsulation dot1q switchport trunk allowed vlan 50,128 switchport mode trunk channel-group 5 mode auto ! interface GigabitEthernet0/52 switchport trunk encapsulation dot1q switchport trunk allowed vlan 50,128 switchport mode trunk channel-group 5 mode auto ! interface Vlan1 ip address 10.80.1.2 255.255.255.0 ! interface Vlan50 ip address 10.80.50.2 255.255.255.0 ! interface Vlan128 ip address 10.80.128.2 255.255.255.0 ip helper-address 10.80.50.6 ip helper-address 10.80.50.8 ! ip classless ip http server ip http secure-server ! ! vstack ! (console and VTY configs removed) ! end -
@tc10284 What I was keying off is this setting on switch 2
interface Vlan1 ip address 10.80.1.2 255.255.255.0 ! interface Vlan50 ip address 10.80.50.2 255.255.255.0 ! interface Vlan128 ip address 10.80.128.2 255.255.255.0 ip helper-address 10.80.50.6 ip helper-address 10.80.50.8remove the IP addresses from the two vlans. I think you need to leave the one IP address on your management vlan but remove the others.
Actually if you just kill the dhcp helper service on that second switch then the second router will quit announcing itself. You only want your main switch forwarding the dhcp packets to your dhcp server(s). You can leave the interface addresses on the vlans, but its not useful.
-
Thanks! I’ll try that shortly, but I won’t be able to test PXE boot again until Wednesday.
-
That appears to have resolved it! Thanks!
