Replication issue between Master node and Storage node

[Hugo]

Server

• FOG Version: 1.5.0-RC-6
• OS: CentOS 7.3

Description

Hello,

I have a FOG master server with two “internal” storage nodes, one mounted on /images and the other on a /images2 repository that i’ve created.

I added an external storage node associated to the primary master node (/images) : Replication works fine

But then I have to add a second external storage node associated to the secondary master node (/images2). When I proceed, I can see it on the dashboard :

In the fogreplicator.log, I can see that the replication start between the two nodes because of the following lines :

And on the side of the /images repository of the node, I can see that folders have been created for each images to replicate :

But unfortunately, it seems that the replication fails during processing because the Node usage disk stays on 100% Free on the dashboard. When I come back to the fogreplicator log after waiting 10 minutes for it to rerun, it appears that the replication is running and doesn’t want to finish :

Any suggestions would be helpful

PS : Sorry for the red frames but I can’t disclose all of the information

[Sebastian Roth]

@Hugo Can’t see an error there yet. Maybe you just have to wait. Either for it to finish or to see an error in the logs…?!

[george1421]

@sebastian-roth I need to fill in a bit of detail here. I helped Hugo setup a partitioned FOG Master server. This is where he has some images in /images and other images in /images2. There is a storage group associated with each images container where the full fog server is the master node in each storage group. From what I gather the storage group that points to /images is replicating just fine to its storage node. But the storage group connected to /images2 is not replicating to its storage node.

@Hugo What we would like to see is a screen shot of “Storage Management -> All Storage Nodes” page. This will tell us if you have the proper grouping right.

We would also like to see your master node configuration for the /images2 container.

[Hugo]

@Sebastian-Roth Yes I can admit that it doesn’t look like an error like this, but I did exactly the same process for the first node I installed. There were about 6 images on it and the replication on the node took about 15 minutes. This new node’s got only 2 images and replication is still running after 30 minutes what makes me say that there’s something wrong.

@george1421 There it is :

[Quazz]

@hugo Can we get the settings for the target node as well?

[Hugo]

@quazz Damn yes you’re right for the bandwidth, I set it to 1 because ONLY for this node, it doesn’ allow me to let it to 0. But when thinking about it that was stupid ahah.

I did that because when I installed the first node, it appears automatically on the GUI with his IP address. Then I just had to rename it adn the bandwidth was automatically set to 0 (no limite if i’m right).
But for the second node, I don’t know why but I have to create it manually with all its settings, and with this method I can’t let the bandwidth to 0… So I let it to 1

Anyway I try to make the installation again with a valid value and keep you in touch

[Hugo]

@Quazz After updating the bandwidth to 10000, I can now update it to 0 in order to remove the limit. The replication is now working fine and I can track it on the disk Usage on the dashboard.

I should have been more careful about the settings before posting … Thanks for your help guys !

[george1421]

@hugo said in Replication issue between Master node and Storage node:

I set it to 1 because ONLY for this node, it doesn’ allow me to let it to 0.

Can you explain this? Is the gui limiting you from turning this off or setting to 0? Did you have to do some trick to set to 0?

I see you are using the new gui in 1.5.0RCx series. I want to make sure there isn’t something in the new gui that is limiting its action.

[Hugo]

@george1421 Yes there’s a little issue on that point : If the Node you just installed is added automatically in the GUI (it appears in the list of storage nodes with its IP) then the bandwidth is set to 0 by default and no problem with that.

But if you have to create the new storage node manually (it was the case for my second one) then you’ve got the following error :

So my trick was to set a value like 10000, update the Node, then reset to 0 manually and Update again.

[george1421]

@Developers Comment please on the 1.5 GUI?

[Tom Elliott]

@george1421 I’ll try to get this fixed, I thought I already did, but I might have missed it already sorry.