FogReplicator and Storage Nodes.

sbenson

@george1421 here is a quick question. Am I supposed to have a default storage node and an extra one for SRO? This might be the problem. The original default storage node was removed so there is only a SRO storage node listed. tho the SRO server has 2 storage nodes. One labeled SyncMHB and one labeled SyncSRO. I will admit the Storage nodes have been extremely confusing from other replication services I have used in the past.

george1421

@sbenson ok that was confusing.

Lets say you have two fog servers (one at hq and one at remote site) and at your remote site you have a storage node (i.e. at your remote site you have 2 fog servers one is a full fog server and one is a storage node). You want images created to HQ to replicate to the other fog servers (full servers or storage nodes).

At your HQ you will create one storage group. That storage group will have your HQ FOG server configured as the default server and your fog server at your remote site. With that configuration your images from the HQ FOG server will replicate to the FOG server at the remote site. You will need to manually create the image definitions to see these images at the remote fog server.

Now at the remote fog server you create a new storage group and add your remote fog server and storage node in that storage group. The remote fog server shall not know about the HQ fog server. Its not part of its configuration. As far as the remote fog server knows, images just magically show up in its /images directory.

george1421

@george1421 The graphic here explains visually what I just stumbled through.
https://forums.fogproject.org/topic/6014/create-the-concept-of-a-foreignmasterstorage-deployment-node/13

sbenson

@george1421 2 Physical servers SRO-FOG-01 and MHB-FOG-01.
for all intents and purposes SRO-FOG-01 is the Main server that will have the majority of work done on it(Creating new images, deployments, changes, etc). The MHB-FOG-01 will deploy images that I would have created on SRO-FOG-01, and synced over to MHB-FOG-01. The MHB-FOG-01 should NEVER tell anyone in the Southern California(MHB-FOG-01) network to pull from the Northern California(SRO-FOG-01) server.

From what you explained it kinda matches what I have…except some things are weird.

SRO-FOG-01 has 2 storage nodes listed, SyncMHB(Not master) and SyncSRO(Master).
…SyncMHB is set to IP x.x.57.42(the IP of MHB-FOG-01), Storage Group default, Img Path = /images, etc…
…SyncSRO is set to the local IP x.x.76.44, Storage Group default, same Image path, etc.

MHB-FOG-01 has one storage node, SRO(Not Master, I was told the name didn’t matter)
…SRO is set to x.x.76.44, default storage group, paths, etc…

It may be best to just start over and say this. I have fog servers in 2 different cities, on 2 different internet circuits. I want the Images on Server 1 to be replicated on Server 2. What is the best method to complete this?

sbenson

@george1421 said in FogReplicator and Storage Nodes.:

https://forums.fogproject.org/topic/6014/create-the-concept-of-a-foreignmasterstorage-deployment-node/13

That is super confusing.

george1421

@sbenson said in FogReplicator and Storage Nodes.:

The MHB-FOG-01 should NEVER tell anyone in the Southern California(MHB-FOG-01) network to pull from the Northern California(SRO-FOG-01) server.

So in this case you should be able to power off the fog server at SRO and the MHB fog server and clients should image without issue. This IS the design.

sbenson

@george1421 said in FogReplicator and Storage Nodes.:

So in this case you should be able to power off the fog server at SRO and the MHB fog server and clients should image without issue. This IS the design.

This is what we were looking for, that is why we wanted it to sync automatically during the middle of the night. Then possibly the only thing we would have to do after a sync is export the images(txt file) on SRO, and import them on the MHB server

george1421

@sbenson said in FogReplicator and Storage Nodes.:

SRO-FOG-01 has 2 storage nodes listed, SyncMHB(Not master) and SyncSRO(Master).
…SyncMHB is set to IP x.x.57.42(the IP of MHB-FOG-01), Storage Group default, Img Path = /images, etc…
…SyncSRO is set to the local IP x.x.76.44, Storage Group default, same Image path, etc.

In this case SRO server should ONLY have the MHB fog server, with the SRO fog server setup as master. All others not the master. This will tell the Master FOG server to replicate to everyone else that is not a master.

On the MHB fog server I think you have one storage node, If that is the case create the MHB fog server as the master node and your MHB storage node as not the master.

This setup will then only sync SRO FOG server -> MHB fog server and MHB Fog server -> MHB Storage node. That way there is only one replication across the WAN.

george1421

@sbenson said in FogReplicator and Storage Nodes.:

@george1421 said in FogReplicator and Storage Nodes.:

https://forums.fogproject.org/topic/6014/create-the-concept-of-a-foreignmasterstorage-deployment-node/13

That is super confusing.

Its super confusing since it is so complex. Your setup is similar to the HQ and ATL setup (right side of the picture). Just remember the FOG server setup as the master in the storage group will send the image to all other nodes not the master node within the storage group.

george1421

@sbenson said in FogReplicator and Storage Nodes.:

Then possibly the only thing we would have to do after a sync is export the images(txt file) on SRO, and import them on the MHB server

Yes this is correct export from SRO and import into MHB.

The developers are working on a way for this setup to automatically update in a future release of fog, maybe 1.4.3 or 1.4.4, but I can’t say. They are creating a new API call that will make this process a bit easier.

sbenson

@george1421 said in FogReplicator and Storage Nodes.:

In this case SRO server should ONLY have the MHB fog server, with the SRO fog server setup as master. All others not the master. This will tell the Master FOG server to replicate to everyone else that is not a master.
On the MHB fog server I think you have one storage node, If that is the case create the MHB fog server as the master node and your MHB storage node as not the master.
This setup will then only sync SRO FOG server -> MHB fog server and MHB Fog server -> MHB Storage node. That way there is only one replication across the WAN.

Sorry for the delay in getting back to you, been busy with other projects. So fog servers pull from the master node, not get pushes from the master node?

So on SRO I create an SRO node where it is the master, it’s own IP and set a user/pass for it
On MHB I set MHB as a master node(storing images in /images?) and a SRO as a client node(also storing images in /images?)

Edit, wait that is totally backwards from what you just said, let me re-work this

Tom Elliott

@sbenson Master nodes push. That’s it.

sbenson

SRO/MASTER

ngmID: 4
       ngmMemberName: SRO
ngmMemberDescription: Local fog storage in SRO
     ngmIsMasterNode: 1
          ngmGroupID: 1
         ngmRootPath: /images
          ngmSSLPath: /opt/fog/snapins/ssl
          ngmFTPPath: /images
       ngmMaxBitrate:
       ngmSnapinPath: /opt/fog/snapins
        ngmIsEnabled: 1
         ngmHostname: x.x.76.44
       ngmMaxClients: 10
   ngmBandwidthLimit: 2000
             ngmUser: sync
             ngmPass: Some PasswordA
              ngmKey:
        ngmInterface: eth0
     ngmGraphEnabled: 1
          ngmWebroot: /fog

MHB slave

        ngmID: 4
       ngmMemberName: MHB
ngmMemberDescription:
     ngmIsMasterNode: 1
          ngmGroupID: 1
         ngmRootPath: /images
          ngmSSLPath: /opt/fog/snapins/ssl
          ngmFTPPath: /images
       ngmMaxBitrate:
       ngmSnapinPath: /opt/fog/snapins
        ngmIsEnabled: 1
         ngmHostname: x.x.57.42
       ngmMaxClients: 1
   ngmBandwidthLimit: 1
             ngmUser: NONVALIDUSER
             ngmPass: Gibberish password
              ngmKey:
        ngmInterface: eth0
     ngmGraphEnabled: 1
          ngmWebroot: /fog
*************************** 2. row ***************************
               ngmID: 5
       ngmMemberName: SRO
ngmMemberDescription:
     ngmIsMasterNode:
          ngmGroupID: 1
         ngmRootPath: /images
          ngmSSLPath: /opt/fog/snapins/ssl
          ngmFTPPath: /images
       ngmMaxBitrate:
       ngmSnapinPath: /opt/fog/snapins
        ngmIsEnabled: 1
         ngmHostname: x.x.76.44
       ngmMaxClients: 10
   ngmBandwidthLimit: 20000
             ngmUser: sync
             ngmPass: Some PasswordA
              ngmKey:
        ngmInterface: eth0
     ngmGraphEnabled: 1
          ngmWebroot: /fog

Does this look correct?

Tom Elliott

@sbenson Well I don’t know what the user/password pair is, but they should be the Linux user with FTP permissions.

george1421

@sbenson no if I’m understanding what I see that is not correct.

On your SRO (master node).

You should have 1 storage group.
That storage group should contain 2 storage nodes.
The first storage node should of course be the SRO node defined as the master.
The second storage node should be MHB fog server. The user ID and password for the MHB server will be found on the MHB fog server in the /opt/fog/.fogsettings file. This user ID and password are required because the master node transfers the images from the master node to the defined storage nodes over FTP. This account is used to log into the remote fog server to start the ftp transfer.
That is all you need for SRO. Once you do that after a short timeout SRO should start replicating the images to MHB (when the fog replicator service has been enabled).

Now on MHB.

You should have one storage group
You should have 2 storage nodes defined in that storage group.
One storage node should be the MHB server configured as a master server.
The second storage node should be the FOG storage node server in MHB.
As soon as your images from SRO exist on MHB and you import the image definitions from SRO, the MHB Fog server will start replicating the images to the storage node in MHB.

It sounds complicated but not really. In each storage group you can only have one master node. Images are only replicated from the master node to all storage nodes in the storage group. Your master node can be members of one or more storage groups.

sbenson

@Tom-Elliott @george1421
I have recreated the nodes, and had someone at the MHB office try and deploy an image(W7P-HP6300). The image was originally created on the SRO server, but synced to the MHB server months ago. I had the machine register with fog, on the MHB(10.63.57.42) server. On a side note the MHB office has 2 network ranges 10.63.57.x and 10.63.65.x). After registering the server, we went in to the MHB fog web gui, set the Image to the W7P-HP6300 image, Tasks, basic, deploy. We then rebooted the machine and selected pxe boot. This started pulling down the image, but a lot slower than we would expect for a gigabit network. This machine was pulling the Image from SRO over our 100mb MPLS network.

We have confirmed that both DHCP pools point to the 10.63.57.42 server.

I dumped the fog database on MHB and the only time 10.63.76.44 is mentioned is in the node defenition previously shown

How does a machine being imaged, boot from the fog server in the same office, register to that server, have an image pushed to it from that server, yet the traffic comes from our main server in SRO?

W7P-HP6300 on MHB

             imageID: 10
           imageName: W7P-HP6300
           imageDesc: Conference Room PC
HP ProDesk 6300
Office
Sysprep
           imagePath: W7PHP6300
        imageProtect: 0
      imageMagnetUri:
       imageDateTime: 2017-05-12 13:28:34
       imageCreateBy: sbenson
       imageBuilding: 0
           imageSize: 104853504.000000:27798433792.000000:
         imageTypeID: 1
imagePartitionTypeID: 1
           imageOSID: 5
         imageFormat:
     imageLastDeploy: 0000-00-00 00:00:00
       imageCompress: 1
        imageEnabled: 1
      imageReplicate: 1

W7P-HP6300 on SRO

             imageID: 5
           imageName: W7P-HP6300
           imageDesc: Conference Room PC
HP ProDesk 6300
Office
Sysprep
           imagePath: W7PHP6300
        imageProtect: 0
      imageMagnetUri:
       imageDateTime: 2016-10-24 21:41:02
       imageCreateBy: sbenson
       imageBuilding: 0
           imageSize: 104853504.000000:6786727.000000:27798433792.000000:
         imageTypeID: 1
imagePartitionTypeID: 1
           imageOSID: 5
         imageFormat:
     imageLastDeploy: 2016-10-24 21:57:14
       imageCompress: 1
        imageEnabled: 1
      imageReplicate: 1

[13:53:20] root@MHB-FOG-01[0]:/opt$ cd fog/
[13:53:21] root@MHB-FOG-01[0]:/opt/fog$ grep 10.63.76.44 -r *
[13:53:31] root@MHB-FOG-01[0]:/opt/fog$ cd /var/www/html/fog/
[13:53:44] root@MHB-FOG-01[0]:/var/www/html/fog$ grep 10.63.76.44 -r *
[13:54:49] root@MHB-FOG-01[0]:/images$ cd W7PHP6300/
[13:55:00] root@MHB-FOG-01[0]:/images/W7PHP6300$ ls -lsa
total 13031508
       4 drwxrwxrwx  2 fog root        4096 Oct 24  2016 .
       4 drwxrwxr-x 14 fog root        4096 May  5 17:08 ..
       4 -rwxrwxrwx  1 fog root           3 Oct 24  2016 d1.fixed_size_partitions
    1024 -rwxrwxrwx  1 fog root     1048576 Oct 24  2016 d1.mbr
       4 -rwxrwxrwx  1 fog root         190 Oct 24  2016 d1.minimum.partitions
       4 -rwxrwxrwx  1 fog root          15 Oct 24  2016 d1.original.fstypes
       0 -rwxrwxrwx  1 fog root           0 Oct 24  2016 d1.original.swapuuids
    9172 -rwxrwxrwx  1 fog root     9390413 Oct 24  2016 d1p1.img
13021288 -rwxrwxrwx  1 fog root 13333793671 Oct 24  2016 d1p2.img
       4 -rwxrwxrwx  1 fog root         190 Oct 24  2016 d1.partitions
[13:55:04] root@MHB-FOG-01[0]:/images/W7PHP6300$ grep 10.63.76.44 d1\.*

george1421

@sbenson said in FogReplicator and Storage Nodes.:

I dumped the fog database on MHB and the only time 10.63.76.44 is mentioned is in the node defenition previously shown

Just to ensure we are on the same page here. MHB FOG server should not have ANY reference to SRO fog server / site or anything. SRO should not be defined in any storage group or storage node on MHB. If this is the case there is nothing to tell any client pxe booting at MHB that the SRO fog server exists.

If you have confirmed this, then we must dig into the MHB fog server configuration because this is an oddity that should never exist.

Tom Elliott

Quick design as I see it in my head:

SRO is “main” server.
MHB is the “slave” server.

SRO Should have a Storage group containing information about both the Main (SRO) and Slave (MHB).

The “Master” node in the “Main” server will be the SRO Node. MHB will just be a part of the group.

All images associated on SRO will only need to be on the SRO Storage Group (this will cause the SRO Server replicate its images to MHB). All that’s needed here is the IP of the MHB and the FOG linux user and password.

The MHB Server will have it’s own group. It also has its own Database.

The MHB Server will also contain its own Node as master. There should not be any need for any other storage nodes on the MHB Server.

The SRO Server should have the Image definitions exported which can then be uploaded to the MHB Server. No need for a database dump as the only group either server will ever have should only be with ID 1 (if nothing has been changed otherwise of course).

This should be all that’s required. When the client machines attempt booting from the MHB network, they should ONLY request the information from the MHB Server and node.

Tom Elliott

sbenson

@Tom-Elliott how does a “storage group” contain information about the nodes? It seems as if each node has to be told it is part of a specific storage group(so it’s a little backward). Also will it cause a problem if the storage groups on both servers are the “Default” or will it require a separate group on each server?

FogReplicator and Storage Nodes.

160

11.6k

17.1k

154.6k