so… as an update…
I stopped/disabled the fog replication service and used the same command as fog used to replicate the folder over from the fog images store to the synology images store.
After this, imaging from the storage node works as desired.
I’d really like to know how fog winds up pulling the Synology home page during the file compare… is it using http? what path is it trying to hit? I can set up a packet trace and try to find out, but I’m hoping someone can just chime in with the info.
What protocol does fog use to verify its images? somehow fog is pulling html from the synology… makes me wonder if it is looking in the /fog web path on the synology?
(the synology guide by george says this folder doesnt matter because its not a real storage node)
wondering if I can ‘create’ a webserver on the synology with the path fog is looking for? just a little more patience guys…i’d appreciate it.
Thanks!
Here are the logs from the replication, if you have time to look at them.
fogreplicator.log https://pastebin.com/Lb0Kc17n
fogimagesize.log https://pastebin.com/NN0nyrbY
fogreplicator…transfer.spg-syn1.log https://pastebin.com/VHtHRZq3
fogreplicator.G3-0118.1.transfer.spg-syn1.log https://pastebin.com/U51QxtkY
Also, just checked with the d1.mbr file… md5 checksums match.
040d5ad57af1942104ab788d3b12778e fogserver
040d5ad57af1942104ab788d3b12778e synology web console
d1p1.img…checksums match here too.
79d8b1a77b8111418481c121c246501b fog server
79d8b1a77b8111418481c121c246501b synology web console
@george1421 yes, the files are making it to the NAS. Fog is replicating at full speed, up to the configured bandwidth limit. Once the files are replicated, the process begins anew.
@wayne-workman Apologies for not providing clearer info. The paths for the storage node are configured within fog as follows:
The ftp paths are as follows:
The files being replicated are within the G3-0118.1 folder, as listed below:
the source folder on the fog server is the /images folder, as shown below.
Additionally, I have verified that both the synololgy and the fog server are using the same ntp source as the rest of the devices on the network. (a windows domain controller). After changing the ntp settings, i stopped the fog replication service, deleted the files from the synology folder, and restarted the replication service.
Thanks. I’ll keep digging. Most likely it is something simple that I have done.
@wayne-workman
yes I did. I logged in via ftp and was able to browse with no issues.
@george1421
Hi George. I had hoped that you would chime in as I followed your fantastic storage node document when I set this up.
in just noticed that the ‘images’ shared folder didn’t have rights for the fog user.
Still doing the same thing. Rolling right through each file over and over again. I changed the rights to the folders as recommended.
ok… I noticed something else… the portions I deleted from the originall fogreplicator.log are html pages from the synology. Is there a reason that the fog server is pulling html files from the synology?
@wayne-workman
missed that part, sorry! I’ll take a shot.
@wayne-workman
I know how to start/stop the service, thanks for that. The problem is that I can’t stop the service fast enough before it starts writing to one of the files again. It is literally continuous replication…over and over again. See below:
2018-02-19 12:35:11 /images/G3-0118.1/d1.fixed_size_partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.fixed_size_partitions 0-5 156b/s
2018-02-19 12:35:11 /images/G3-0118.1/d1.mbr -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.mbr 0-1048576 15.15M/s
2018-02-19 12:35:11 /images/G3-0118.1/d1.minimum.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.minimum.partitions 0-330 6.7K/s
2018-02-19 12:35:12 /images/G3-0118.1/d1.original.fstypes -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.original.fstypes 0-40 1.9K/s
2018-02-19 12:35:12 /images/G3-0118.1/d1.original.swapuuids -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.original.swapuuids 0-0
2018-02-19 12:35:12 /images/G3-0118.1/d1.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.partitions 0-330 21.5K/s
2018-02-19 12:35:41 /images/G3-0118.1/d1.fixed_size_partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.fixed_size_partitions 0-5 455b/s
2018-02-19 12:35:41 /images/G3-0118.1/d1.mbr -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.mbr 0-1048576 14.29M/s
2018-02-19 12:35:41 /images/G3-0118.1/d1.minimum.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.minimum.partitions 0-330 40.3K/s
2018-02-19 12:35:41 /images/G3-0118.1/d1.original.fstypes -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.original.fstypes 0-40 5.6K/s
2018-02-19 12:35:41 /images/G3-0118.1/d1.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.partitions 0-330 46.0K/s
2018-02-19 12:36:08 /images/G3-0118.1/d1p1.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p1.img 0-364722448 12.87M/s
2018-02-19 13:32:10 /images/G3-0118.1/d1p2.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p2.img 0-44072908270 12.50M/s
2018-02-19 13:48:51 /images/G3-0118.1/d1p3.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p3.img 0-13123585448 12.50M/s
2018-02-19 13:48:53 /images/G3-0118.1/d1p4.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p4.img 0-34532488 17.37M/s
2018-02-19 13:56:11 /images/G3-0118.1/d1.fixed_size_partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.fixed_size_partitions 0-5 500b/s
2018-02-19 13:56:11 /images/G3-0118.1/d1.mbr -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.mbr 0-1048576 19.23M/s
2018-02-19 13:56:11 /images/G3-0118.1/d1.minimum.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.minimum.partitions 0-330 46.0K/s
2018-02-19 13:56:11 /images/G3-0118.1/d1.original.fstypes -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.original.fstypes 0-40 4.9K/s
2018-02-19 13:56:11 /images/G3-0118.1/d1.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.partitions 0-330 46.0K/s
2018-02-19 13:56:45 /images/G3-0118.1/d1p1.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p1.img 0-364722448 10.35M/s
2018-02-19 14:52:47 /images/G3-0118.1/d1p2.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p2.img 0-44072908270 12.50M/s
2018-02-19 15:09:28 /images/G3-0118.1/d1p3.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p3.img 0-13123585448 12.50M/s
2018-02-19 15:09:30 /images/G3-0118.1/d1p4.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p4.img 0-34532488 12.55M/s
2018-02-19 15:16:06 /images/G3-0118.1/d1.fixed_size_partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.fixed_size_partitions 0-5 455b/s
2018-02-19 15:16:06 /images/G3-0118.1/d1.mbr -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.mbr 0-1048576 18.87M/s
2018-02-19 15:16:06 /images/G3-0118.1/d1.minimum.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.minimum.partitions 0-330 46.0K/s
2018-02-19 15:16:06 /images/G3-0118.1/d1.original.fstypes -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.original.fstypes 0-40 5.6K/s
2018-02-19 15:16:06 /images/G3-0118.1/d1.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.partitions 0-330 46.0K/s
2018-02-19 15:16:33 /images/G3-0118.1/d1p1.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p1.img 0-364722448 12.84M/s
@wayne-workman
Also, the fogreplicator…transfer.(synologyname).log shows the following:
(edited for brevity…it just flips the same two files back and forth)
2018-02-19 13:55:42 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 24.3K/s
2018-02-19 14:05:42 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 17.7K/s
2018-02-19 14:05:44 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 4.4K/s
2018-02-19 14:15:42 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 19.1K/s
2018-02-19 14:15:43 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 8.1K/s
2018-02-19 14:25:42 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 3.6K/s
2018-02-19 14:25:43 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 24.3K/s
2018-02-19 14:35:42 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 5.7K/s
2018-02-19 14:35:44 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 4.6K/s
2018-02-19 14:45:43 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 5.5K/s
2018-02-19 14:45:43 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 24.3K/s
2018-02-19 14:55:42 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 5.1K/s
2018-02-19 14:55:44 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 4.2K/s
2018-02-19 15:05:43 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 2.8K/s
2018-02-19 15:05:44 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 24.3K/s
Thanks for the quick reply.
Plenty of space… (9TB free)
Here are the abbreviated contents of the fogreplicator.log:
[02-19-18 7:55:43 pm] | Files do not match.
[02-19-18 7:55:43 pm] * Deleting remote file: /images/dev/postinitscripts/fog.postinit
[02-19-18 7:55:43 pm] * Starting Sync Actions
[02-19-18 7:55:43 pm] | CMD:
lftp -e 'set xfer:log 1; set xfer:log-file "/opt/fog/log/fogreplicator..transfer.spg-syn1.log";set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; set net:limit-rate 0:13107200; mirror -c -r -R --ignore-time -vvv --exclude ".srvprivate" "/images/dev/postinitscripts" "/images/dev/postinitscripts"; exit' -u foguser,[Protected] 10.1.0.133
[02-19-18 7:55:43 pm] * Started sync for Image dev/postinitscripts
[02-19-18 7:55:43 pm] * Not syncing Image between groups
[02-19-18 7:55:43 pm] | Image Name: G3-0118.1
[02-19-18 7:55:43 pm] | There are no other members to sync to.
[02-19-18 7:55:43 pm] * Attempting to perform Group -> Nodes image replication.
[02-19-18 7:55:43 pm] * Found Image to transfer to 1 node
[02-19-18 7:55:43 pm] | Image Name: G3-0118.1
[02-19-18 7:55:43 pm] | Replication already running with PID: 9840
[02-19-18 8:05:40 pm] * Starting Image Replication.
[02-19-18 8:05:40 pm] * We are group ID: 1. We are group name: default
[02-19-18 8:05:40 pm] * We are node ID: 1. We are node name: DefaultMember
[02-19-18 8:05:40 pm] * Attempting to perform Group -> Group image replication.
[02-19-18 8:05:40 pm] | Replicating postdownloadscripts
[02-19-18 8:05:40 pm] * Found Image to transfer to 1 node
[02-19-18 8:05:40 pm] | File Name: postdownloadscripts
[02-19-18 8:05:41 pm] | 235 235 /images/postdownloadscripts/fog.postdownload <!DOCTYPE html>
....(file contents here)....
[02-19-18 8:05:41 pm] | Files do not match.
[02-19-18 8:05:41 pm] * Deleting remote file: /images/postdownloadscripts/fog.postdownload
[02-19-18 8:05:42 pm] * Starting Sync Actions
[02-19-18 8:05:42 pm] | CMD:
lftp -e 'set xfer:log 1; set xfer:log-file "/opt/fog/log/fogreplicator..transfer.spg-syn1.log";set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; set net:limit-rate 0:13107200; mirror -c -r -R --ignore-time -vvv --exclude ".srvprivate" "/images/postdownloadscripts" "/images/postdownloadscripts"; exit' -u foguser,[Protected] 10.1.0.133
[02-19-18 8:05:42 pm] * Started sync for Image postdownloadscripts
[02-19-18 8:05:42 pm] | Replicating postinitscripts
[02-19-18 8:05:42 pm] * Found Image to transfer to 1 node
[02-19-18 8:05:42 pm] | File Name: dev/postinitscripts
[02-19-18 8:05:43 pm] | 249 249 /images/dev/postinitscripts/fog.postinit <!DOCTYPE html>
....(file contents printed here)....
[02-19-18 8:05:43 pm] | Files do not match.
[02-19-18 8:05:43 pm] * Deleting remote file: /images/dev/postinitscripts/fog.postinit
[02-19-18 8:05:43 pm] * Starting Sync Actions
[02-19-18 8:05:43 pm] | CMD:
lftp -e 'set xfer:log 1; set xfer:log-file "/opt/fog/log/fogreplicator..transfer.spg-syn1.log";set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; set net:limit-rate 0:13107200; mirror -c -r -R --ignore-time -vvv --exclude ".srvprivate" "/images/dev/postinitscripts" "/images/dev/postinitscripts"; exit' -u foguser,[Protected] 10.1.0.133
[02-19-18 8:05:43 pm] * Started sync for Image dev/postinitscripts
[02-19-18 8:05:43 pm] * Not syncing Image between groups
[02-19-18 8:05:43 pm] | Image Name: G3-0118.1
[02-19-18 8:05:43 pm] | There are no other members to sync to.
[02-19-18 8:05:43 pm] * Attempting to perform Group -> Nodes image replication.
[02-19-18 8:05:43 pm] * Found Image to transfer to 1 node
[02-19-18 8:05:43 pm] | Image Name: G3-0118.1
[02-19-18 8:05:43 pm] | Replication already running with PID: 9840Hi all! Let me first say how awesome Fog is and thank you for making it so amazing.
I am having an issue with replication between my primary FOG node (running centos 7) and a remote storage node, running on a synology system. I have followed the synology storage node guides and have everything set up at the remote site. It all works perfectly except that the main node continually replicates the image files to the secondary node. Not sure why. Any ideas?
Thanks!