Setting up FOG in multi-location environment
-
Hi,
We’re trying to change our setup from just a bunch of unrelated FOG servers on multiple sites to a setup that automatically replicates images and snap-ins from the central site to satellite sites.
To accomplish this we have installed the Location (and Site + Access Controls to limit what’s visible to satellite fog users) Plug-in, installed all satellite site FOG servers as Storage Nodes to the central site Master Node and attached each node to a separate location.
At the central site we have a score of VLANS, all hosts are on different VLANS from the Master Node. At satellite sites we have hosts on two VLANS, where one is the same as the storage node VLAN. Routing is setup and allows multicasting, ip pim is set on the VLANs the server is not on.
Directed broadcast ACL permitting the local storage node for WoL is set for all routed VLANs at all sites.
Hosts are attached to locations and sites corresponding to their physical locations.
Replication is working fine. However, multicasting is not working at satellite sites. Clients just hang at the blue imaging screen after receiving the task. Unicasting is working.
WoL is not working at satellite sites VLANs different from the the Storage Node VLAN. Wireshark shows that no magic packets are sent to the routed VLAN, only on the same VLAN as the Storage Node, even though it is defined in the WoL Broadcasts menu. It is as if the Location Plug-in is not reaching in to the WoL Broadcast menu for the satellite node to be able to send magic packets to the correct VLAN?
Both multicasting and WoL worked with current network settings before changing from full FOG servers to Storage Nodes.
To try and get multicating to work at satellite sites, we tried adding separate Storage Groups and assigning each satellite node to their own Storage Group, setting each node as Master Node for their corresponding group, adding the Storage Groups to the images and snap-ins as described in this post:
https://forums.fogproject.org/topic/11653/multi-cast-with-location-plugin-enabled/18?_=1673903176664
That didn’t do the trick, though.Is it even possible to do it this way? I suppose we could fix both issues by making the satellite nodes fully fledged FOG servers with different Storage Groups and then assigning multiple Storage Groups to images and snap-ins?
That would leave us with a separate DB and user administration at each site, though.
We’re running FOG 1.5.9 on CentOS 7.9 at all sites. Central site is a physical server, satellites sites are virtual servers on ESXi 7.
Any help appreciated - Thanks.
-
This post is deleted! -
@tag said in Setting up FOG in multi-location environment:
Is it even possible to do it this way?
What you’ve tried is what I was going to suggest. Having a storage group per site, each setup as a master storage node. Do you have any error messages?
-
Thanks for replying.
First of all - we have it working now.
Regarding multicasting the node logs showed an error about the interface not being ready for some reason. The age old secret and very esoteric IT-tech hack known as “server reboot” fixed that issue. I don’t know what the issue was as such as server access, unicasting, deploying snap-ins, dhcpd (everything else, really) all worked fine. I guess we should end the process of changing from standard install to FOG node with a reboot from now on.
As for WoL we wiresharked the VLANS and found that the node DID in fact send magic packets for hosts on remote VLANS. It turned out that IOS-XE v17 is buggy and needs an extra setting on the ingress interface, i.e. the interface on which the server or node sends the magic packets. You need to set “ip network broadcast” as well as the usual “ip directed broadcast <ACL#>”. We have identical routers at all sites (Cisco 1111-4P), but some are from a previous purchase an thus has IOS-XE v16. Here WoL works when setup as described here:
https://wiki.fogproject.org/wiki/index.php/WOL_Forwarding
So maybe an addition to the wiki would be helpful to others doing WoL forwarding.Anyway, thanks for taking your time to confirm we were doing it the right way.
-
@tag Thanks for the update. Great to hear you were able to find and fix the issues. Adding this piece of information to the Wiki is a good idea. Though we are in the process of moving that to http://docs.fogproject.org and I am wondering if have a little bit of spare time to help on this?
-
[[undefined-on, S Sebastian Roth, ]]
-
@Sebastian-Roth Sure, happy to help in any way I can…
