FOG With more than 1 subnet
-
@igorpa2 You have a lot of info here so lets try to unpack it into a truth table as such:
- Does it pxe boot correctly on the local subnet and remote subnet?
- Does it image correctly on the local subnet and the remote subnet.
FOG is only designed to image using a single IP address, period. When you install FOG it asks you what interface do you want to use for imaging and then configures itself for that. If you put two network adapters in the FOG server it will still use that single IP address you defined when fog was installed. Now I keep saying single IP address, because it IS allowed to team multiple network adapter together, still having a single IP address but multiple interfaces to spread the bandwidth among.
So lets first start with, does both uefi and bios computers boot to the FOG menu on the local subnet and the remote subnet?
-
-
With the fog with two network adapter, they boot only on legacy clients. On UEFI clients, only on the subnet that have configured on installation process they boot. With the FOG with one network adapter + one FOG Storage Node configured in other subnet, both uefi and legacy pxe boot.
-
With the fog with two network adapters, yes, but when i put the FOG Master installation in one subnet and the FOG Storage Node in another subnet, no. What I wrote in the previous post happens, where for example, a client that is on network 172, tried to use network node 200 and, a client that was on network 200, tried to use network node 172 causing rate problems transmission as mentioned above.
-
-
@igorpa2 Again I wan to focus on PXE booting first then imaging, the storage node solution works but the discussion is a bit more in depth.
What is the purpose of the second network adapter in the FOG server? Its OK to have it for system management but not for imaging.
Your description is still confusing because I don’t know the design of your network. Is both subnets at the same location or are they in two different locations (cities for example). Or is everything on one campus?
-
@george1421 No no, it’s on same campus. We have one LAN network with more 3 Vlan’s. We have FOG installed on LAN interface(200.x.x.x) and we added another interface(172.x.x.x), because when on clients on subnet 172 network, we have encountered some problems. I don’t remember anymore if are pxe boot or imaging problem we have encountered because we installed in 2018 =P . I can test again and try to use fog only with one interface and see if the boot and imaging works with only the LAN interface on both subnets(200 and 172).
-
@igorpa2 Ok for pxe booting to work you need to have a dhcp server on each subnet or a central one that sends the IP address of the fog server in dhcp option 66 and the boot loader in dhcp option 67. For bios computers your dhcp server needs to send undionly.kpxe and for uefi computers the dhcp server need to send ipxe.efi or snp.efi boot loaders.
So for your campus, what device is your dhcp server? Is it a windows 2012 or later box or linux, or your FOG server acting as a dhcp server?
-
@george1421 We have one PfSense as DHCP Server. We have options 66 and 67 configured on dhcp server of each subnet. undionly.kkpxe for bios and snponly.efi for uefi.
-
@igorpa2 OK very good. PFSense does work very well for pxe booting with FOG. It has both fields for uefi and bios as you noted.
So have you confirmed that pxe booting works correctly for both uefi and bios on each subnet? Are you sure you are not using the non-imaging interface for anything at the moment? Lets just focus on the interface that was defined when fog was installed. As you noted you can review the /opt/fog/.fogsettings file to see how the questions were answered when fog was installed.
-
@george1421 said in FOG With more than 1 subnet:
So have you confirmed that pxe booting works correctly for both uefi and bios on each subnet?
With our actual scenario, yes.(One master fog in 200 subnet with one network interface on it and another fog installed as storage node on 172 subnet with one network interface).
-
@igorpa2 OK then so what I understand then we can discount anything regarding tftp from your previous post.
So to the imaging point. You have two storage nodes. One is the master node and one is a slave node. In your FOG configuration are they in the same storage group? If yes, did you install the the Location plugin into the FOG server and assign each storage node to a location?
-
@george1421 said in FOG With more than 1 subnet:
So to the imaging point. You have two storage nodes. One is the master node and one is a slave node. In your configuration are they in the same storage group?
Yes
@george1421 said in FOG With more than 1 subnet:
If yes, did you install the the Location plugin into the FOG server and assign each storage node to a location?
No, i don’t have installed the location plugin.
-
@igorpa2 said in FOG With more than 1 subnet:
No, i don’t have installed the location plugin.
Without the location plugin the clients will typically image from the master node until the client count is reached then the next target computer will roll over to the slave node.
So go and install the location plugin. Create your two locations. They can be called anything just they must be named differently. Finally assign a storage node to a location.
-
@george1421 said in FOG With more than 1 subnet:
@igorpa2 said in FOG With more than 1 subnet:
No, i don’t have installed the location plugin.
Without the location plugin the clients will typically image from the master node until the client count is reached then the next target computer will roll over to the slave node.
So go and install the location plugin. Create your two locations. They can be called anything just they must be named differently. Finally assign a storage node to a location.
Okay done, installed, created and assigned.
-
@igorpa2 Ok the last bit of setup for the location plugin is to assign target computers to the location that way they know which storage node is their home server.
Once you do that the pxe booting computer will contact the master node during pxe boot, load ipxe then find out which storage node to use to pull the image from.
Just be aware the way fog works, you can only capture images to the master node. Slave nodes are deploy only nodes.
So the linkage is storage node to a location and target computer to a location so they can both find each other.
-
@george1421 said in FOG With more than 1 subnet:
@igorpa2 Ok the last bit of setup for the location plugin is to assign target computers to the location that way they know which storage node is their home server.
Once you do that the pxe booting computer will contact the master node during pxe boot, load ipxe then find out which storage node to use to pull the image from.
Just be aware the way fog works, you can only capture images to the master node. Slave nodes are deploy only nodes.
So the linkage is storage node to a location and target computer to a location so they can both find each other.
Well done, i have configured the hosts. Thank you George for help, tomorrow i will test if will work. Just to know, it’s possible to use only the Master FOG installed on 200 network with one network interface and this work to all others subnets ? I will test this environment again to see if i will have problems and/or to remember what problems i had when on i installed the FOG for first time. Thank you!
-
@igorpa2 said in FOG With more than 1 subnet:
Just to know, it’s possible to use only the Master FOG installed on 200 network with one network interface and this work to all others subnets
Yes this is how I have it setup on my campus is that just one fog server and one interface can image all 6 vlans. Understand that imaging across your vlans will put a network load on your vlan router which may impact your overall transfer rates. Imaging a fog server on a 1GbE network on the same vlan as the fog server, you should see transfer rates (according to partclone) in the 5.5 to 6.2GB/min using contemporary target computer as a baseline. Across your subnets I would expect in the lower 5GB/min range. Now my infrastructure used 10GbE in the core with a 10GbE router and I see 13-14GB/min to target computers attached to an access layer switch at 1GbE.
-
@george1421 said in FOG With more than 1 subnet:
@igorpa2 said in FOG With more than 1 subnet:
Just to know, it’s possible to use only the Master FOG installed on 200 network with one network interface and this work to all others subnets
Yes this is how I have it setup on my campus is that just one fog server and one interface can image all 6 vlans. Understand that imaging across your vlans will put a network load on your vlan router which may impact your overall transfer rates. Imaging a fog server on a 1GbE network on the same vlan as the fog server, you should see transfer rates (according to partclone) in the 5.5 to 6.2GB/min using contemporary target computer as a baseline. Across your subnets I would expect in the lower 5GB/min range. Now my infrastructure used 10GbE in the core with a 10GbE router and I see 13-14GB/min to target computers attached to an access layer switch at 1GbE.
Yes, in our 1GbE network, we see 5 to 7GB/min transfer rate, like your network, but when client connect on node that are on another subnet(client on 172 subnet coneccting on 200 node) the transfer don’t exceed 2GB/min and some cases reaching a whopping 20~50MB/Min. I honestly don’t know why this happens, but I’ll check all the settings again and test using how you use them and see what problems we have. I will update this thread again with any conclusions I reach.
Thank you.
-
@igorpa2 We do have some tools built into the FOS Linux (the os that runs on the target computer) where we can put FOS Linux in debug mode and then test network throughput to see if the network links are able to pass 1GbE or not if you want to do some debugging. My bet is that your vlan router can’t maintain the normal traffic flow plus add imaging traffic. In some testing I’ve done, I can flood a 1GbE link on a server with just 3 unicast images running at the same time. 6.1GB/min equates to about 100MB/s or about 1Gb/s (the full bandwidth of a 1GbE link. Understand that number in partclone is actually the entire data path and not just network, so its a bit misleading. But know we have tools like iperf3 on FOS Linux so we can test bandwidth back to the fog server for debugging slow connections.
-
@george1421 I have tested to use only one FOG to use on all subnets and I have a problem that I can’t transmit the file via TFTP. My client on network 172 normally receives the IP from DHCP, but does not receive the file to boot.
I also did the following test: I entered my FOG from network 172, and used the TFTP get command for my main FOG server and got the same error when I try to boot with the 172 clients. The file does not come, it gives a timeout error . Do you have any idea what it could be? In clients from network 200, the boot occur normally.
-
@igorpa2 said in FOG With more than 1 subnet:
entered my FOG from network 172, and used the TFTP get command for my main FOG server and got the same error
I have see something similar but not with subnets on the same campus. I have seen this with a WAN configuration, where the MTU of the link is below the block size of tftp and the packets get fragmented and then discarded by the WAN router.
Lets rule out network connectivity.
- Can you ping the FOG server on the 200 vlan from the 172 vlan?
- Is there some type of screening router or firewall between the two vlans that might filter out tftp traffic?
-
- Yes, i can ping each other on both two subnets.
- Yes, pfsense itself. I set the rules to pass all ports and all protocols between the FOG IP Server and the 172 network.
The MTU is in blank, what seems to be the default 1500.
