• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

Continuous Fog Storage Node Replication problem

Scheduled Pinned Locked Moved
FOG Problems
4
32
3.7k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • M
    MPSAdmin
    last edited by Feb 20, 2018, 9:28 PM

    Also, just checked with the d1.mbr file… md5 checksums match.

    040d5ad57af1942104ab788d3b12778e fogserver
    040d5ad57af1942104ab788d3b12778e synology web console

    d1p1.img…checksums match here too.
    79d8b1a77b8111418481c121c246501b fog server
    79d8b1a77b8111418481c121c246501b synology web console

    G 2 Replies Last reply Feb 20, 2018, 9:33 PM Reply Quote 0
    • G
      george1421 Moderator @MPSAdmin
      last edited by Feb 20, 2018, 9:33 PM

      @mpsadmin OK then we might need an opinion from one of the @developers to give us an idea why the NAS is not reporting what the FOG replicator expects.

      If I remember right there is 2 or 3 replicator logs. One is the main replicator log, one is for the storage node, and there may be a log file for each image being replicated. I don’t have access to my production FOG server at the moment to confirm. But I thought the logs were pretty descriptive to why it was replicating.

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

      1 Reply Last reply Reply Quote 0
      • G
        george1421 Moderator @MPSAdmin
        last edited by george1421 Feb 20, 2018, 3:51 PM Feb 20, 2018, 9:50 PM

        @mpsadmin I have my fog-pi server with me that has FOG 1.4.4 on it. I’m looking at the code and it appears to rely on the file byte count being the same to indicate if the file needs to be replicated or not. I’m not a programmer so this object oriented php is a bit greek to me. But so far it looks like byte count is the trigger. Now this is the byte count as reported by FTP session. Also the log writes to /opt/fog/log look fairly descriptive to what exactly the replicator is doing.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

        1 Reply Last reply Reply Quote 0
        • W
          Wayne Workman @MPSAdmin
          last edited by Feb 20, 2018, 11:16 PM

          @mpsadmin Did you use the FTP option when logging in via WinSCP? I ask because I want to be sure. Photo:
          0_1519168575571_winscp ftp.png

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
          Daily Clean Installation Results:
          https://fogtesting.fogproject.us/
          FOG Reporting:
          https://fog-external-reporting-results.fogproject.us/

          M 1 Reply Last reply Feb 21, 2018, 2:32 PM Reply Quote 0
          • W
            Wayne Workman @MPSAdmin
            last edited by Wayne Workman Feb 20, 2018, 5:19 PM Feb 20, 2018, 11:18 PM

            @mpsadmin said in Continuous Fog Storage Node Replication problem:

            the portions I deleted from the originall fogreplicator.log are html pages from the synology. Is there a reason that the fog server is pulling html files from the synology?

            @Developers I believe that quote is a symptom of the root problem. I am just trying to drill down to why.

            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
            Daily Clean Installation Results:
            https://fogtesting.fogproject.us/
            FOG Reporting:
            https://fog-external-reporting-results.fogproject.us/

            T 1 Reply Last reply Feb 20, 2018, 11:23 PM Reply Quote 0
            • T
              Tom Elliott @Wayne Workman
              last edited by Feb 20, 2018, 11:23 PM

              @wayne-workman It’s not a problem fog is causing. It’s a problem with how fog manages to do things. The request being made to the synology is not a fog based system and so synology is responding with its own information.

              This is just a guess on my part, but it’s unlikely to be anything we can fix.

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

              1 Reply Last reply Reply Quote 0
              • M
                MPSAdmin @Wayne Workman
                last edited by Feb 21, 2018, 2:32 PM

                @wayne-workman yes. used ftp.
                0_1519223532615_12c40913-36db-419e-9802-bdbafed4fdf8-image.png

                1 Reply Last reply Reply Quote 0
                • M
                  MPSAdmin
                  last edited by Feb 21, 2018, 2:56 PM

                  Here are the logs from the replication, if you have time to look at them.

                  fogreplicator.log https://pastebin.com/Lb0Kc17n
                  fogimagesize.log https://pastebin.com/NN0nyrbY
                  fogreplicator…transfer.spg-syn1.log https://pastebin.com/VHtHRZq3
                  fogreplicator.G3-0118.1.transfer.spg-syn1.log https://pastebin.com/U51QxtkY

                  1 Reply Last reply Reply Quote 0
                  • M
                    MPSAdmin
                    last edited by Feb 22, 2018, 1:49 PM

                    What protocol does fog use to verify its images? somehow fog is pulling html from the synology… makes me wonder if it is looking in the /fog web path on the synology?

                    (the synology guide by george says this folder doesnt matter because its not a real storage node)

                    wondering if I can ‘create’ a webserver on the synology with the path fog is looking for? just a little more patience guys…i’d appreciate it.

                    Thanks!

                    G 1 Reply Last reply Feb 22, 2018, 2:08 PM Reply Quote 0
                    • G
                      george1421 Moderator @MPSAdmin
                      last edited by Feb 22, 2018, 2:08 PM

                      @mpsadmin My intent was to set this back up in my lab but day to day work has got in the way.

                      From looking at the code it appears that the fog server replication connects to the storage node over FTP and pulls the target file size. If they are not the same then the replicator starts over.

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                      W 1 Reply Last reply Feb 22, 2018, 2:41 PM Reply Quote 0
                      • W
                        Wayne Workman @george1421
                        last edited by Wayne Workman Feb 22, 2018, 8:42 AM Feb 22, 2018, 2:41 PM

                        @george1421 said in Continuous Fog Storage Node Replication problem:

                        it appears that the fog server replication connects to the storage node over FTP and pulls the target file size.

                        I don’t think that’s how it’s doing it. I think it’s making an HTTP call to a PHP script on the storage nodes to get file size. And that’s why it’s outputting an entire webpage instead of a size (because the NAS is redirecting the call to it’s landing page). The FOG Image Replicator is supposed to revert to FTP for filesize if the HTTP call doesn’t work. I believe it’s only checking the HTTP status code and not verifying that the output is a single number - so that’s why an entire web page is being pushed to the replication logs - and why it’s not reverting to FTP for filesize. @developers

                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                        Daily Clean Installation Results:
                        https://fogtesting.fogproject.us/
                        FOG Reporting:
                        https://fog-external-reporting-results.fogproject.us/

                        1 Reply Last reply Reply Quote 0
                        • M
                          MPSAdmin
                          last edited by Feb 22, 2018, 2:42 PM

                          so… as an update…
                          I stopped/disabled the fog replication service and used the same command as fog used to replicate the folder over from the fog images store to the synology images store.

                          After this, imaging from the storage node works as desired.

                          I’d really like to know how fog winds up pulling the Synology home page during the file compare… is it using http? what path is it trying to hit? I can set up a packet trace and try to find out, but I’m hoping someone can just chime in with the info.

                          1 Reply Last reply Reply Quote 0
                          • 1
                          • 2
                          • 1 / 2
                          1 / 2
                          • First post
                            30/32
                            Last post

                          219

                          Online

                          12.0k

                          Users

                          17.3k

                          Topics

                          155.2k

                          Posts
                          Copyright © 2012-2024 FOG Project