Fog 1.6 Multiple Sites, Location not working as intended
-
Fog 1.6.0.14
OS: Ubuntu 24.04
Hardware: VMsI have several locations that I am currently testing the latest release of Fog 1.6.0.14. I have the location plugin on, and by sites setup with corresponding Storage Groups and Storage Nodes. The Main Fog Server with DB is at Site A, A storage node is set to master at Site B. Everything is replicated but when deploying an Image at site B, it will always pull from Site A, unless the Image has its “Image Primary Storage Group” changed to Site B from Site A. After making this change if I deploy at site A then the Image will pull from site B instead of Site A. The images needed to be accessible from all sites at all times but pull from the specific location rather than across our uplink.
-
@FlareImp I tested this on 1.6.0.14’s code with the location plugin. A host in a location whose storage group is not the image’s primary group gets that location’s group and node, at task creation and at boot. So the plugin does pick the local node when the host has a location.
In 1.6, “Site” and “Location” are two separate things. A Site (Sites menu) only controls which users can see and manage which hosts. It has no effect on where an image comes from. A Location (location plugin) is what picks the storage group and node.
Please check these for one Site B host:
- Open the host, go to its Location tab. Is it set to the Site B location?
- Open that location. Which storage group and which storage node does it show?
- Is the image listed in the Site B storage group (Image > Storage Groups tab)?
If the host has no location, it uses the image’s primary group, which is what you see.
-
@Tom-Elliott My Apologies, when I say site, I’m referring to the location where my fog VMs are at, not what is within the Fog WebGUI.
Open the host, go to its Location tab. Is it set to the Site B location?
All 31 Computers are set to “Site B”Open that location. Which storage group and which storage node does it show?
The Storage Group is “SiteB” and the Storage Node is “SiteB-Test”Is the image listed in the Site B storage group (Image > Storage Groups tab)?
The Image is associated with “Site A” and “Site B” -
When I deploy the image it will show the correct storage node under tasks up until the point of actual deployment, then it switches back to the default location of the Main Fog Server.
-
@FlareImp Thank you, that detail found it. This is a bug in the 1.6 installer, not your setup.
What happens: a storage node install does not download the plugins, so your Site B node has no location plugin in its web root. Your Site B clients PXE boot from that node, so its boot.php answers them. Without the plugin, it falls back to the image’s primary storage group and changes the task to that node. That is the switch you see when the deploy starts.
The fix: https://github.com/FOGProject/fogproject/pull/2050. It is on dev-branch now and reaches stable in the next stable release.
You have two ways to get going before then.
Option 1: copy the plugins to the node (stays on 1.6.0.14). Run this on the Site B node as root:
rsync -a root@<main-server-ip>:/var/www/html/fog/lib/plugins/ /var/www/html/fog/lib/plugins/Then deploy again. A reinstall of the storage node removes the copy, so repeat it after each storage node install until you are on the fixed release.
Option 2: move to dev-branch (tests the real fix). Update the main server FIRST, then the storage node. Do not update only the storage node: dev-branch has a newer database schema than 1.6.0.14, and a storage node never updates the schema. Until the main server updates it, the node sends every PXE boot to the schema update page instead of the boot menu.
On the main server, then on the Site B node, from your fogproject checkout:
cd /path/to/fogproject/bin sudo ./updatefog.sh --branch dev-branchOn the main server, answer Y when the installer asks to update the database schema. Finish the main server before you start the node.
--branchis a one-off: the next plainupdatefog.shrun goes back to the stable channel.Either way, please let me know if the Site B deploy now pulls from SiteB-Test.
-
@Tom-Elliott Option 1 and doing a reboot of the site B node has taken care of the issue. Thanks for your awesomness!
I have come across another problem though. Would you like a new thread started or just add it here?
-
@FlareImp What’s the issue?
-
@Tom-Elliott I assigned an image to site B and it starts to replicate but then after a time it will stop coping and the folder will disappear from the /images directory on the storage node. Then once the replication service kicks off again it starts all over again. I didn’t notice this before because I manually ran rsync to mirror what was in the images directory on the main fog server. I have to remote site B set to throttle at 200mb since I don’t want to saturate the pipe.
