FOG Image Replication questions

[FallingWax]

Running Version 1.3.4
SVN Revision: 6066

I have been dealing with issues with the Image Replication process blowing up my networks. I have a large number of images that need to transfer down to two storage nodes. I gave up on the one and brought it back in house from the remote location and synced up the images. The other unit I have is set with most of the images but around 100 GB of image data still needs to transfer across the net. Whenever the replication service starts it goes full bore and wrecks the local network to the point that its unusable. I want to configure image replication to only run between 5 pm and 8 am. I think i figured this part out but my question is in the behavior of the service. If the service stops mid transfer of an image when it starts up again, does it continue where it left off? From what i can tell, it does not. It sees that something did not complete, deletes it from the node and starts over again. If some can clarify the process it would be make this set up easier to solve.

In addition to this, am I correct that this process is done strictly through ftp? I can change the QoS on my firewall to throttle down traffic on a specific port but I am using 21 for other applicationa as well. If i change the default ftp port in the GUI do I need to change it anywhere else? That would solve my issue as well.

Any help is appreciated.

Thanks!

[Wayne Workman]

@FallingWax said in FOG Image Replication questions:

If the service stops mid transfer of an image when it starts up again, does it continue where it left off?

I found a bug in lftp about a year ago (was already known about, I just re-discovered it). The bug affects lftp’s ability to mirror properly. The solution at the time then was - if the file size or first 10MB hash doesn’t match with what it should be - delete the destination file and start over. So yeah, it just starts over. Perhaps the lftp bug is fixed now, I’m not sure. If it is, Tom can set it to mirror the differences instead of just starting over. Mirroring differences was how it was before we found the bug.

am I correct that this process is done strictly through ftp?

lftp, which uses ftp. You can set throttling for replication right inside of the FOG Web Interface, see the picture below.

[FallingWax]

@Wayne-Workman Thanks! Can you tell me, if I change the default ftp port in the web gui, is there anywhere else I need to change it?

[Wayne Workman]

@FallingWax if your running centos 7 per wiki instructions you would need to open the new port on the receiving servers with firewalld and possibly configure ftp on the receiving servers to listen to the new port.

Also keep in mind the worst that can happen is replication breaks and image captures don’t complete the last step ( moving the image from /images/dev to /images via ftp ). Kernel updating via the Web interface could also be affected - but that feature isn’t something that’s heavily used.

The risks are relatively low. You know what it is your changing. Just change it and try stuff out.

[FallingWax]

@Wayne-Workman Good point. Thanks for your help