Setting up FOG 1.2.0 in Multiple VLANs

[Christian Nichols]

[B]Environment Details:[/B]
[LIST]
[]School District Network with each school on their own VLAN
[]Windows Server 2008 R2 w/ AD, DHCP, and DNS
[]FOG server installed on Ubuntu Server 12.04.5 LTS
[]FOG server (hardware server) hooked into trunked VLAN port in switch
[]DHCP 066 is set to “fogserver” and not a specific IP
[]Configured DNS Forward Lookup Zones to point “fogserver” to it’s relative VLAN IP
[LIST]
[*][IMG]http://i.imgur.com/t2GZ5SS.png[/IMG]
[/LIST]
[/LIST]
[B]FOG Installation Configuration:[/B]
[CODE] Here are the settings FOG will use:
Base Linux: Debian
Detected Linux Distribution: Ubuntu
Installation Type: Normal Server
Server IP Address: 10.52.4.23
DHCP router Address: 10.52.4.20
DHCP DNS Address: 10.52.4.20
Interface: eth2
Using FOG DHCP: 0
Internationalization: 0
Donate: 0

    DHCP will NOT be setup but you must setup your
    current DHCP server to use FOG for PXE services.

    On a Linux DHCP server you must set:
        next-server

    On a Windows DHCP server you must set:
        option 066 & 067

      Option 066 is the IP of the FOG Server: (e.g. 10.52.4.23)
      Option 067 is the undionly.kpxe file: (e.g. undionly.kpxe)

[/CODE]
[B]**NOTE: [/B]I figure maybe when FOG asks for IP’s like above, it may cause issues since 10.52.4.x is the High School VLAN. The only way a server/workstation can communicate with another server/workstation on another VLAN is to use the DNS name I configured in DNS Forward Lookup Zones (“fogserver” in my case) but the FOG Install Script only wants IP addresses.

[B]/etc/network/interfaces configuration:[/B]
[CODE]# The loopback network interface
auto lo
iface lo inet loopback

The primary network interface

auto eth2

iface eth2 inet dhcp

High School

auto eth2.4
iface eth2.4 inet static
address 10.52.4.23
netmask 255.255.252.0
gateway 10.52.4.1
dns-nameservers 10.52.4.20 8.8.8.8
vlan-raw-device eth2

Junior High

auto eth2.8
iface eth2.8 inet static
address 10.52.8.23
netmask 255.255.252.0
dns-nameservers 10.52.8.20 8.8.8.8
vlan-raw-device eth2

Central Office

auto eth2.10
iface eth2.10 inet static
address 10.52.1.23
netmask 255.255.255.0
dns-nameservers 10.52.1.20 8.8.8.8
vlan-raw-device eth2

Primary School

auto eth2.12
iface eth2.12 inet static
address 10.52.12.23
netmask 255.255.252.0
dns-nameservers 10.52.12.20 8.8.8.8
vlan-raw-device eth2

Elementary School

auto eth2.16
iface eth2.16 inet static
address 10.52.16.23
netmask 255.255.252.0
dns-nameservers 10.52.16.20 8.8.8.8
vlan-raw-device eth2
[/CODE]

[B]RESULTING PROBLEM:[/B]
[B][IMG]http://i.imgur.com/34wfJco.png[/IMG][/B]

Any suggestions?

[ArchFan]

Have you ever had FOG running in this configuration, or is this a first attempt?

[Mentaloid]

I don’t see anything particularly wrong so far… I use DNS hostnames for DHCP option 66 on my network. The tftp is failing before undionly.kpxe is loaded via tftp, so it’s not a FOG configuration issue (server ip during setup).

Have you verified tftp works from that client location already?

[Tom Elliott]

Are your switches configured to point back at the dhcp server for each of the different subnets/vlans?

Many times, when one has multiple VLANs as you do, the DHCP is still handled centrally. However, in order for those subnets to communicate across one another, they need to know how to pass the data back and forth. This is where the DHCP server really shines.

One thing I notice, however, is that the separate VLANs all have their own DNS servers? Do each of the VLANs have their own Gateway address as well?

We have 12 separate VLANs in our organization. All have their own Gateway’s that lead back to our routing switch. Our routing switch has an ip-helper setting back to our central DHCP server for their related Gateway. This way there’s still a gate to communicate across, and the routing switch has all the routes configured to allow pass of traffic from one subnet to another.

I think this may be where your network is having issues.

[Christian Nichols]

[B]@ArchyFan [/B]I had 0.32 working fine before our network was VLAN’d so this is the first attempt at trying to get FOG working in a VLAN’d environment.

[B]@Mentaloid[/B] TFTP test fails from client location “Timeout occurred”

[B]@Tom[/B]
[QUOTE]Are your switches configured to point back at the dhcp server for each of the different subnets/vlans?[/QUOTE]
I [I]believe[/I] they are. How would I be sure? (Forgive me for my ignorance, I didn’t configure the switches; however, I can gain access to them).

[QUOTE]Many times, when one has multiple VLANs as you do, the DHCP is still handled centrally. However, in order for those subnets to communicate across one another, they need to know how to pass the data back and forth. This is where the DHCP server really shines.[/QUOTE]
I [I]believe[/I] this is how our’s is setup (handled centrally). The pic may be entirely irrelevant to the problem but extra info anyways:
[IMG]http://i.imgur.com/NjFXNTY.png[/IMG]

[QUOTE]One thing I notice, however, is that the separate VLANs all have their own DNS servers? Do each of the VLANs have their own Gateway address as well?[/QUOTE]
They all have the same physical DNS server (the Windows Server - also our DC, DHCP, and DNS server). The different DNS IP’s on each VLAN all point back to the same server. The gateway IP’s are also different relative to their respected VLAN; however, they all point to same router/firewall.

[IMG]http://i.imgur.com/sLJ7qZS.png[/IMG]

[QUOTE]We have 12 separate VLANs in our organization. All have their own Gateway’s that lead back to our routing switch. Our routing switch has an ip-helper setting back to our central DHCP server for their related Gateway. This way there’s still a gate to communicate across, and the routing switch has all the routes configured to allow pass of traffic from one subnet to another.[/QUOTE]
Would you know how I would setup an ip-helper setting like you’re talking about? I’ll research Fortinet’s website for it. Is that called a [I]DHCP relay[/I]?

Thanks fellas!

[George]

Hi

I don t think that it is something with your vlans. In the photo that you have send, client is getting 10.52.5.35 and your dhcp server is 10.52.4.20. I think that this is not correct. Another thing that you should try is to configure the dhcp option 66 with the ip of the fog server and see what happens.

[Christian Nichols]

[quote=“George, post: 36267, member: 1565”]Hi

I don t think that it is something with your vlans. In the photo that you have send, client is getting 10.52.5.35 and your dhcp server is 10.52.4.20. I think that this is not correct. Another thing that you should try is to configure the dhcp option 66 with the ip of the fog server and see what happens.[/quote]

Thanks for your reply, George.

The client is connected to the High School VLAN which DHCP offers 10.52.4-5.x so it is correct in that sense. If I configured 066 to be an IP (10.52.4.23) then only High School VLAN would be able to see it and boot to PXE.

[Tom Elliott]

Have you tried applying the scope options for all of the individual vlan’s rather than just on the server?

[George]

[quote=“Christian Nichols, post: 36275, member: 2195”]Thanks for your reply, George.

The client is connected to the High School VLAN which DHCP offers 10.52.4-5.x so it is correct in that sense. If I configured 066 to be an IP (10.52.4.23) then only High School VLAN would be able to see it and boot to PXE.[/quote]

As Tom said you should configure option 66 in every scope that you have made and for me use the ip not dns. For network 1.0 you should use 1.23 for option 66.

[George]

Can you tell me something else? In the dhcp do you have different interface for every vlan or you are using dhcp relay. it is strange that ipxe client is seeing the dhcp ip from another network

[Christian Nichols]

[quote=“Tom Elliott, post: 36277, member: 7271”]Have you tried applying the scope options for all of the individual vlan’s rather than just on the server?[/quote]

I had no idea I could configure options 66 & 67 in every scope. I thought you could only do it under Server Options like in my screenshot.

I’ll mess with it tomorrow to see if it works

[Christian Nichols]

You guys were right! I’m booting into PXE and can see the FOG menu now! Ah I’m so excited
[IMG]http://i.imgur.com/l32robu.png[/IMG]

Yep… didn’t know you could configure options for individual scopes. Today I learned.

I’m going to try to import all of my FOG 0.32 Host info and images now and see where I stand then. Thank you guys so much for helping me out - I would’ve never figured it out without the assistance.

[Christian Nichols]

[B]UPDATE: [/B]
I was receiving [QUOTE]FATAL: INT18: BOOT FAILURE[/QUOTE] message when selecting [I]Boot to Disk[/I], resolved it by updating to the latest Published Kernel (x64)

[Christian Nichols]

Well now that I’ve actually had the chance to venture out to the workstations on the other VLANs I noticed all of them are booting to PXE but once the FOG Menu is suppose to appear it errors out and reboots immediately. The error message flashes too quickly for me to tell what the problem was. It boots to the FOG menu fine if the workstation is located within the High School VLAN scope ([B]10.52.4-5.x[/B])

I’m going to assume it’s FOG’s [B]TFTP[/B] IP ([B]10.52.4.23[/B]) in FOG Settings > TFTP Server. The workstations on the [B]other[/B] VLANs cannot communicate with that IP because it is a High School VLAN IP.
[IMG]http://i.imgur.com/7nlVWb7.png[/IMG]

I may try manually changing FOG’s TFTP IP (FOG Settings > TFTP Server)to the FOG IP relevant to that specific VLAN to see if it will work - worse case, I guess, would be having to do that each time you want to use FOG on another VLAN

[Tom Elliott]

On each of your switches, I’m going to guess that you’ve got STP (Spanning Tree Protocol) enabled? Can you enable Portfast or Rapid STP?

[Christian Nichols]

[quote=“Tom Elliott, post: 36343, member: 7271”]On each of your switches, I’m going to guess that you’ve got STP (Spanning Tree Protocol) enabled? Can you enable Portfast or Rapid STP?[/quote]

I will check on this as soon as I get credentials for the switches. We outsourced the VLAN configuration and never got the credentials.

Will update soon.

[Christian Nichols]

Okay I’ve obtained the credentials and have access via telnet and web interface. Haven’t tried console.

Questions before I start configuring the switches:
[LIST]
[]STP is enabled. Does it need to be disabled or just enable Portfast or Rapid STP alongside it?
[]Can this be done on just one of the end switches (like a lab switch) to see if it fixes the problem or does it have to be enabled on every switch in the network for it to have any effect?
[]Should this be done after hours or do you think it can safely be done during the work day to avoid a lot of downtime for end-users?
[]Do you have a link to a tutorial/thread explaining how to do this with Cisco switches? Thought I’d ask.
[/LIST]
Thanks!

[Tom Elliott]

STP, I’m assuming is enabled for a reason, so if you can I’d recommend start by enabling Portfast/Rapid STP if you can. If you cannot, see about disabling stp throughout if you can.

You can do it, for testing, on a switch at a time, so yes, you can “test” by enabling/disabling as needed to a known problem area.

I don’t know how your switches react, so I’d say, just for performance, if you can test it after hours.

I don’t have a tutorial, I’m sorry.

[Junkhacker]

i seem to recall someone having a short writeup about cisco switch configuration on the forums, or at least a link to one. some forum searching might yield useful results

[Christian Nichols]

[quote=“Junkhacker, post: 36537, member: 21583”]i seem to recall someone having a short writeup about cisco switch configuration on the forums, or at least a link to one. some forum searching might yield useful results[/quote]
I found it. Jaymes wrote one. Thanks

[url]http://fogproject.org/forum/threads/cisco-ws-c2960s-not-passing-pxe-or-proxydhcp.9916/[/url]

[quote=“Tom Elliott, post: 36535, member: 7271”]STP, I’m assuming is enabled for a reason, so if you can I’d recommend start by enabling Portfast/Rapid STP if you can. If you cannot, see about disabling stp throughout if you can.

You can do it, for testing, on a switch at a time, so yes, you can “test” by enabling/disabling as needed to a known problem area.

I don’t know how your switches react, so I’d say, just for performance, if you can test it after hours.

I don’t have a tutorial, I’m sorry.[/quote]

Okay, I’ve enabled PortFast on all of the workstation interfaces (and on also on FOG Server switch port) and I’ve changed the Switch Mode from PVST to Rapid-PVST on both the FOG server switch & workstation switch with the same outcome. It doesn’t seem to make a difference. Connection timeout when it tries to load /default.ipxe

However, it [B]DOES[/B] work on workstations connected to the same switch as the FOG Server.

NOTE: I haven’t rebooted the switches since I’ve made those changes. Would that make a difference maybe?

Any suggestions?