• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

Continuous Fog Storage Node Replication problem

Scheduled Pinned Locked Moved
FOG Problems
4
32
3.7k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • W
    Wayne Workman @MPSAdmin
    last edited by Wayne Workman Feb 19, 2018, 2:15 PM Feb 19, 2018, 8:14 PM

    @mpsadmin So in the replication log, it should print the command the replicator is using for each of these replication activities.

    You can stop the replicator like this generally:

    systemctl stop FOGImageReplicator
    

    And then try to manually run the command from the master node. See what happens as it runs. You will need to input the correct password in the command.

    Later, you can turn replication back on like this:

    systemctl start FOGImageReplicator
    

    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
    Daily Clean Installation Results:
    https://fogtesting.fogproject.us/
    FOG Reporting:
    https://fog-external-reporting-results.fogproject.us/

    M 1 Reply Last reply Feb 19, 2018, 8:22 PM Reply Quote 0
    • M
      MPSAdmin @Wayne Workman
      last edited by Feb 19, 2018, 8:19 PM

      @wayne-workman
      Also, the fogreplicator…transfer.(synologyname).log shows the following:
      (edited for brevity…it just flips the same two files back and forth)

      2018-02-19 13:55:42 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 24.3K/s
      2018-02-19 14:05:42 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 17.7K/s
      2018-02-19 14:05:44 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 4.4K/s
      2018-02-19 14:15:42 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 19.1K/s
      2018-02-19 14:15:43 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 8.1K/s
      2018-02-19 14:25:42 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 3.6K/s
      2018-02-19 14:25:43 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 24.3K/s
      2018-02-19 14:35:42 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 5.7K/s
      2018-02-19 14:35:44 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 4.6K/s
      2018-02-19 14:45:43 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 5.5K/s
      2018-02-19 14:45:43 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 24.3K/s
      2018-02-19 14:55:42 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 5.1K/s
      2018-02-19 14:55:44 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 4.2K/s
      2018-02-19 15:05:43 /images/postdownloadscripts/fog.postdownload -> ftp://foguser@10.1.0.133/images/postdownloadscripts/fog.postdownload 0-235 2.8K/s
      2018-02-19 15:05:44 /images/dev/postinitscripts/fog.postinit -> ftp://foguser@10.1.0.133/images/dev/postinitscripts/fog.postinit 0-249 24.3K/s

      1 Reply Last reply Reply Quote 0
      • M
        MPSAdmin @Wayne Workman
        last edited by Feb 19, 2018, 8:22 PM

        @wayne-workman
        I know how to start/stop the service, thanks for that. The problem is that I can’t stop the service fast enough before it starts writing to one of the files again. It is literally continuous replication…over and over again. See below:

        2018-02-19 12:35:11 /images/G3-0118.1/d1.fixed_size_partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.fixed_size_partitions 0-5 156b/s
        2018-02-19 12:35:11 /images/G3-0118.1/d1.mbr -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.mbr 0-1048576 15.15M/s
        2018-02-19 12:35:11 /images/G3-0118.1/d1.minimum.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.minimum.partitions 0-330 6.7K/s
        2018-02-19 12:35:12 /images/G3-0118.1/d1.original.fstypes -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.original.fstypes 0-40 1.9K/s
        2018-02-19 12:35:12 /images/G3-0118.1/d1.original.swapuuids -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.original.swapuuids 0-0
        2018-02-19 12:35:12 /images/G3-0118.1/d1.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.partitions 0-330 21.5K/s
        2018-02-19 12:35:41 /images/G3-0118.1/d1.fixed_size_partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.fixed_size_partitions 0-5 455b/s
        2018-02-19 12:35:41 /images/G3-0118.1/d1.mbr -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.mbr 0-1048576 14.29M/s
        2018-02-19 12:35:41 /images/G3-0118.1/d1.minimum.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.minimum.partitions 0-330 40.3K/s
        2018-02-19 12:35:41 /images/G3-0118.1/d1.original.fstypes -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.original.fstypes 0-40 5.6K/s
        2018-02-19 12:35:41 /images/G3-0118.1/d1.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.partitions 0-330 46.0K/s
        2018-02-19 12:36:08 /images/G3-0118.1/d1p1.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p1.img 0-364722448 12.87M/s
        2018-02-19 13:32:10 /images/G3-0118.1/d1p2.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p2.img 0-44072908270 12.50M/s
        2018-02-19 13:48:51 /images/G3-0118.1/d1p3.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p3.img 0-13123585448 12.50M/s
        2018-02-19 13:48:53 /images/G3-0118.1/d1p4.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p4.img 0-34532488 17.37M/s
        2018-02-19 13:56:11 /images/G3-0118.1/d1.fixed_size_partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.fixed_size_partitions 0-5 500b/s
        2018-02-19 13:56:11 /images/G3-0118.1/d1.mbr -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.mbr 0-1048576 19.23M/s
        2018-02-19 13:56:11 /images/G3-0118.1/d1.minimum.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.minimum.partitions 0-330 46.0K/s
        2018-02-19 13:56:11 /images/G3-0118.1/d1.original.fstypes -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.original.fstypes 0-40 4.9K/s
        2018-02-19 13:56:11 /images/G3-0118.1/d1.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.partitions 0-330 46.0K/s
        2018-02-19 13:56:45 /images/G3-0118.1/d1p1.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p1.img 0-364722448 10.35M/s
        2018-02-19 14:52:47 /images/G3-0118.1/d1p2.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p2.img 0-44072908270 12.50M/s
        2018-02-19 15:09:28 /images/G3-0118.1/d1p3.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p3.img 0-13123585448 12.50M/s
        2018-02-19 15:09:30 /images/G3-0118.1/d1p4.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p4.img 0-34532488 12.55M/s
        2018-02-19 15:16:06 /images/G3-0118.1/d1.fixed_size_partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.fixed_size_partitions 0-5 455b/s
        2018-02-19 15:16:06 /images/G3-0118.1/d1.mbr -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.mbr 0-1048576 18.87M/s
        2018-02-19 15:16:06 /images/G3-0118.1/d1.minimum.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.minimum.partitions 0-330 46.0K/s
        2018-02-19 15:16:06 /images/G3-0118.1/d1.original.fstypes -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.original.fstypes 0-40 5.6K/s
        2018-02-19 15:16:06 /images/G3-0118.1/d1.partitions -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1.partitions 0-330 46.0K/s
        2018-02-19 15:16:33 /images/G3-0118.1/d1p1.img -> ftp://foguser@10.1.0.133/images/G3-0118.1/d1p1.img 0-364722448 12.84M/s

        W 1 Reply Last reply Feb 19, 2018, 8:23 PM Reply Quote 0
        • W
          Wayne Workman @MPSAdmin
          last edited by Feb 19, 2018, 8:23 PM

          @mpsadmin The idea was to stop the service - wait for all the replicators to stop - then try to execute the LFTP command manually so you can observe any error messages or undesired behavior.

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
          Daily Clean Installation Results:
          https://fogtesting.fogproject.us/
          FOG Reporting:
          https://fog-external-reporting-results.fogproject.us/

          M 1 Reply Last reply Feb 19, 2018, 8:25 PM Reply Quote 0
          • M
            MPSAdmin @Wayne Workman
            last edited by Feb 19, 2018, 8:25 PM

            @wayne-workman
            missed that part, sorry! I’ll take a shot.

            1 Reply Last reply Reply Quote 0
            • M
              MPSAdmin
              last edited by Feb 19, 2018, 8:35 PM

              ok… I noticed something else… the portions I deleted from the originall fogreplicator.log are html pages from the synology. Is there a reason that the fog server is pulling html files from the synology?

              W 2 Replies Last reply Feb 19, 2018, 8:37 PM Reply Quote 0
              • W
                Wayne Workman @MPSAdmin
                last edited by Feb 19, 2018, 8:37 PM

                @mpsadmin said in Continuous Fog Storage Node Replication problem:

                Is there a reason that the fog server is pulling html files from the synology?

                It could be that the FTP Root is not set correctly for the synology. See this section: https://wiki.fogproject.org/wiki/index.php?title=Troubleshoot_FTP#FTP_Path There are notes in there specifically about Synology devices. Use the troubleshooting higher in the article to determine what the FTP Root path is.

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                Daily Clean Installation Results:
                https://fogtesting.fogproject.us/
                FOG Reporting:
                https://fog-external-reporting-results.fogproject.us/

                1 Reply Last reply Reply Quote 0
                • G
                  george1421 Moderator
                  last edited by Feb 19, 2018, 9:37 PM

                  Here is my POC tutorial to setup a synology nas as a FOG storage node: https://forums.fogproject.org/topic/9430/synology-nas-as-fog-storage-node

                  You need to watch directory permission on the synology NAS as well as nfs share permissions.

                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                  M 1 Reply Last reply Feb 19, 2018, 10:28 PM Reply Quote 1
                  • M
                    MPSAdmin
                    last edited by Feb 19, 2018, 10:18 PM

                    Still doing the same thing. Rolling right through each file over and over again. I changed the rights to the folders as recommended.

                    W 1 Reply Last reply Feb 19, 2018, 10:26 PM Reply Quote 0
                    • W
                      Wayne Workman @MPSAdmin
                      last edited by Feb 19, 2018, 10:26 PM

                      @mpsadmin But did you check the FTP root by logging in via FTP?

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                      Daily Clean Installation Results:
                      https://fogtesting.fogproject.us/
                      FOG Reporting:
                      https://fog-external-reporting-results.fogproject.us/

                      M 1 Reply Last reply Feb 19, 2018, 10:33 PM Reply Quote 0
                      • M
                        MPSAdmin @george1421
                        last edited by Feb 19, 2018, 10:28 PM

                        @george1421
                        Hi George. I had hoped that you would chime in as I followed your fantastic storage node document when I set this up.
                        in just noticed that the ‘images’ shared folder didn’t have rights for the fog user.

                        G 1 Reply Last reply Feb 19, 2018, 10:36 PM Reply Quote 0
                        • M
                          MPSAdmin @Wayne Workman
                          last edited by Feb 19, 2018, 10:33 PM

                          @wayne-workman
                          yes I did. I logged in via ftp and was able to browse with no issues.

                          W 1 Reply Last reply Feb 19, 2018, 11:28 PM Reply Quote 0
                          • G
                            george1421 Moderator @MPSAdmin
                            last edited by george1421 Feb 19, 2018, 4:37 PM Feb 19, 2018, 10:36 PM

                            @mpsadmin So are the files actually making it to the NAS? The only reason why the fog server would keep replicating is because the check sums would not match so it will send the file over again. Possibly date/time mismatch might trigger the replication again, but I think they are focusing on the check sum as the key to replicate.

                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                            M 1 Reply Last reply Feb 20, 2018, 8:40 PM Reply Quote 0
                            • W
                              Wayne Workman @MPSAdmin
                              last edited by Feb 19, 2018, 11:28 PM

                              @mpsadmin said in Continuous Fog Storage Node Replication problem:

                              I logged in via ftp and was able to browse with no issues.

                              That’s not what I am referring to - I would like for you to FTP into the NAS and figure out where you land - i.e. the FTP Path directory.

                              • Is the directory where you land the FTP Path that is set for this node?
                              • What is your landing directory?

                              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                              Daily Clean Installation Results:
                              https://fogtesting.fogproject.us/
                              FOG Reporting:
                              https://fog-external-reporting-results.fogproject.us/

                              M 1 Reply Last reply Feb 20, 2018, 8:37 PM Reply Quote 0
                              • M
                                MPSAdmin @Wayne Workman
                                last edited by Feb 20, 2018, 8:37 PM

                                @wayne-workman Apologies for not providing clearer info. The paths for the storage node are configured within fog as follows:
                                0_1519158734380_c31cc54a-37a2-40d7-aa84-1ab1f8a791b6-image.png

                                The ftp paths are as follows:
                                0_1519158795996_966faeda-2106-434a-b85e-3dcbc6c308f2-image.png

                                The files being replicated are within the G3-0118.1 folder, as listed below:
                                0_1519158844529_806dca24-017a-4760-ac1f-1448da4e5ba3-image.png

                                the source folder on the fog server is the /images folder, as shown below.
                                0_1519158945084_de15469e-5417-453b-937a-52daf3e9da66-image.png

                                Additionally, I have verified that both the synololgy and the fog server are using the same ntp source as the rest of the devices on the network. (a windows domain controller). After changing the ntp settings, i stopped the fog replication service, deleted the files from the synology folder, and restarted the replication service.

                                Thanks. I’ll keep digging. Most likely it is something simple that I have done.

                                W 1 Reply Last reply Feb 20, 2018, 11:16 PM Reply Quote 0
                                • M
                                  MPSAdmin @george1421
                                  last edited by Feb 20, 2018, 8:40 PM

                                  @george1421 yes, the files are making it to the NAS. Fog is replicating at full speed, up to the configured bandwidth limit. Once the files are replicated, the process begins anew.

                                  1 Reply Last reply Reply Quote 0
                                  • M
                                    MPSAdmin
                                    last edited by Feb 20, 2018, 9:28 PM

                                    Also, just checked with the d1.mbr file… md5 checksums match.

                                    040d5ad57af1942104ab788d3b12778e fogserver
                                    040d5ad57af1942104ab788d3b12778e synology web console

                                    d1p1.img…checksums match here too.
                                    79d8b1a77b8111418481c121c246501b fog server
                                    79d8b1a77b8111418481c121c246501b synology web console

                                    G 2 Replies Last reply Feb 20, 2018, 9:33 PM Reply Quote 0
                                    • G
                                      george1421 Moderator @MPSAdmin
                                      last edited by Feb 20, 2018, 9:33 PM

                                      @mpsadmin OK then we might need an opinion from one of the @developers to give us an idea why the NAS is not reporting what the FOG replicator expects.

                                      If I remember right there is 2 or 3 replicator logs. One is the main replicator log, one is for the storage node, and there may be a log file for each image being replicated. I don’t have access to my production FOG server at the moment to confirm. But I thought the logs were pretty descriptive to why it was replicating.

                                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                      1 Reply Last reply Reply Quote 0
                                      • G
                                        george1421 Moderator @MPSAdmin
                                        last edited by george1421 Feb 20, 2018, 3:51 PM Feb 20, 2018, 9:50 PM

                                        @mpsadmin I have my fog-pi server with me that has FOG 1.4.4 on it. I’m looking at the code and it appears to rely on the file byte count being the same to indicate if the file needs to be replicated or not. I’m not a programmer so this object oriented php is a bit greek to me. But so far it looks like byte count is the trigger. Now this is the byte count as reported by FTP session. Also the log writes to /opt/fog/log look fairly descriptive to what exactly the replicator is doing.

                                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                        1 Reply Last reply Reply Quote 0
                                        • W
                                          Wayne Workman @MPSAdmin
                                          last edited by Feb 20, 2018, 11:16 PM

                                          @mpsadmin Did you use the FTP option when logging in via WinSCP? I ask because I want to be sure. Photo:
                                          0_1519168575571_winscp ftp.png

                                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                          Daily Clean Installation Results:
                                          https://fogtesting.fogproject.us/
                                          FOG Reporting:
                                          https://fog-external-reporting-results.fogproject.us/

                                          M 1 Reply Last reply Feb 21, 2018, 2:32 PM Reply Quote 0
                                          • 1
                                          • 2
                                          • 2 / 2
                                          2 / 2
                                          • First post
                                            14/32
                                            Last post

                                          220

                                          Online

                                          12.0k

                                          Users

                                          17.3k

                                          Topics

                                          155.2k

                                          Posts
                                          Copyright © 2012-2024 FOG Project