Image sharing between sites
-
@george1421 Thank you so much for your quick reply. I’ll definitely look into this. Thou I may not be able to do so until next week.
-
@george1421 said in Image sharing between sites:
@KennethJ said in Image sharing between sites:
But what we would like the possibility to do, is create a image at one of these sites and then send it to the others, and then have it automatically available in that FOG-server. Do you have any feature for something like this? An autosync or manuall image transfer?
There is an unsupported feature in FOG. You can do this with one caveat.
How to do this: On your master fog server (or the one where you are doing your image development work). Create a storage group (or use the existing default storage group), add the master fog server as the master of that storage group. Then manually add the other remote Normal FOG servers as storage nodes in that storage group.
Once that is done, any images with the
Replicate?(in the image definition) that is enabled will be replicated to all storage nodes in the active storage group. This will get the raw image files to the remote fog server.I’ve tried setting up two FOG servers as you said. But images marked with the
Replicate?seem to not be replicated to the storage node. Does it need a restart or something to start replicating? Both the Master-node and the remote-node are set up with aReplication Bandwidth (Kbps)of 100,000, andMax Clients= 10. Do they need to have the sameInterface? Another possible issue might be with our network setup. Our network is setup in a way that makes it impossible for the remote-node to contact the master-node. Could this be an issue? All access within our network is done top-down. -
@KennethJ said in Image sharing between sites:
Our network is setup in a way that makes it impossible for the remote-node to contact the master-node
Top down is fine as long as the master node can make http calls as well as ftp to the remote site. Remember that ftp is a two way street the master will open a command channel to the remote FTP server, and the remote FTP server will open a data channel back to the master FOG server (ftp client) to send the file over. This should be allowed through your site to site links since ftp is a well known communications protocol that has these 2 channels. If this is a limitation for your network design, you will loose a little flexibility but you could use rsync (outside of FOG) to send files from the master node to the remote storage nodes you will loose the ability to control when an image is replicated but you can achieve this top down communications.
There are FOG replicator logs in /opt/fog/logs that might give you an understanding why replication isn’t working too. I would begin with the logs then see where that leads you.
-
@george1421 Should this work between all versions of fog? If one server is 1.5.4 and one is 1.3.5 for instance?
-
@KennethJ That I’m afraid is too big of a gap in versions.
Replication may work somewhat. The files should be replicated to the remote nodes, but replication will happen over and over since the remote node doesn’t have the ability to answer the current file size. That was added/changed in version 1.5.5.
As a test (if we think its ftp), from your master fog server can you (from the linux command prompt) ftp to the remote fog server, with the user
fogand the password taken from the remote fog servers configuration file/opt/fog/.fogsettings. In that file there is a field password= you need that password to remote in via ftp. Once logged in change to the /images directory and create a test directory. That should test the connection. Just a note, you will need to use this password (from the remote fog server’s .fogsettings file) when you setup the slave node settings on the master FOG servers’s storage group. If you can’t login via ftp command line then this is where you need to check. -
@KennethJ As well there was a change in the hash sum check between 1.5.4 and 1.5.5! As a general advice I’d stick to use one version only. If you want proper support and bug fixes then using the most current (1.5.6 as of now) is the only way to go.
-
@george1421 Have tried a bit now and it seems the FTP port is closed by default on all our networks. Is it possible to make FOG use a different port for the FTP-connection? We do not want to open the default FTP port.
-
@KennethJ Are you aware of FTP not just using a single port?! I know FTP is not that great and I even have thought about moving on to use a different protocol for replication but it’s a lot of work to change that and I don’t seem to get to it any time soon.
So unfortunately I don’t think there is an easy way to change this right now. The best I could think of is using a tunnel (e.g. VPN) between the sites and allow FTP traffic only through the tunnel.
-
@KennethJ said in Image sharing between sites:
Have tried a bit now and it seems the FTP port is closed by default on all our networks
Is ssh port open? While FOG doesn’t currently support it, you may need to use rsync in place of FOG Replication to sync your image files. You will loose some capability with this option, but you might bypass the ftp port blockage.
-
@Sebastian-Roth Ah, I was not aware of that. If that is the case it seems the rsync @george1421 suggested might be our best option, and probably the one we will try to use. Do you have any suggestions on how to do this @george1421 or is the “migrate fog” wiki artikle our best bet?
-
@KennethJ So I’ve got the image sharing between the sites working. But is there any way to do the export/import of the image through a script? Or a way to create the ‘image_export.csv’ file from a terminal? Or can this only be done using the ‘export image’ function from the fog manager?
-
@KennethJ Perhaps the API? https://news.fogproject.org/simplified-api-documentation/
-
@Quazz Thank you. I’ll look into the API.
This issue could probably be closed now. The image “export/import through API”-issue seems to me is a new and separate, though closely connected, issue.
For future readers the best solution, if FTP is possible over your network, would be to set up the remote servers as a “storage node” with “Is Master Node” unchecked. When this is done you simply check the “Replicate?” field for the images you want to be synchronized across all sites. You will still need to use the import/export functionality to make the images available at the remote sites.
However if FTP is not possible over your network you need to use rsync or scp or your preferred file transfer method, to transfer the images. (More examples can be found here: https://wiki.fogproject.org/wiki/index.php?title=Migrate_images_manually) This also requires the use of the import/export functionality.
