• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Possible Image and Snapin Replication Problem w/ Working Branch

    Scheduled Pinned Locked Moved Solved
    Bug Reports
    5
    29
    6.9k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J
      Jim Graczyk @Wayne Workman
      last edited by

      @wayne-workman

      I did the tail command you posted with ’ | grep error’ on the end. Here’s what matched:

      [root@Sal1FOGV1 bin]# tail -n 500 /var/log/httpd/error_log | grep error
      [Mon Sep 04 12:06:49.751559 2017] [:error] [pid 110308] [client 192.168.100.171:49805] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Malwarebytes.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Mon Sep 04 12:25:33.023607 2017] [:error] [pid 110150] [client 192.168.100.171:49721] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Mon Sep 04 13:00:58.193160 2017] [:error] [pid 86532] [client 192.168.100.171:49742] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Malwarebytes.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Mon Sep 04 13:02:51.022016 2017] [:error] [pid 93233] [client 192.168.100.171:49766] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Office_v2013_PP.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Tue Sep 05 14:31:15.976881 2017] [:error] [pid 89324] [client 192.168.100.196:50414] PHP Warning: ftp_chmod(): SITE CHMOD command failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 137
      [Tue Sep 05 15:00:08.147464 2017] [:error] [pid 86534] [client 10.179.100.156:49891] PHP Warning: array_filter() expects parameter 1 to be array, null given in /var/www/html/fog/lib/fog/image.class.php on line 164, referer: http://fogserver/fog/management/index.php?node=image&sub=membership&id=23
      [Tue Sep 05 15:30:25.281071 2017] [:error] [pid 86534] [client 10.179.100.177:58270] PHP Warning: ftp_chmod(): SITE CHMOD command failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 137
      [Tue Sep 05 16:20:21.716391 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
      [Tue Sep 05 16:20:21.716447 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_delete(): Delete operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 244
      [Tue Sep 05 16:20:21.717251 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
      [Tue Sep 05 16:20:21.717739 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
      [Tue Sep 05 16:20:21.718304 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
      [Tue Sep 05 16:20:21.718808 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
      [Tue Sep 05 16:20:21.719324 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
      [Tue Sep 05 16:20:21.719736 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
      [Tue Sep 05 16:20:21.720228 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
      [Tue Sep 05 16:20:21.720734 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
      [Tue Sep 05 16:20:21.868833 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_chmod(): SITE CHMOD command failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 137
      [Tue Sep 05 22:18:12.376965 2017] [:error] [pid 6299] [client 192.168.100.196:49729] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Tue Sep 05 22:23:05.179099 2017] [:error] [pid 6299] [client 192.168.100.196:49690] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Office_v2013_Std.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Wed Sep 06 10:06:51.808428 2017] [:error] [pid 101284] [client 192.168.100.196:50025] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Wed Sep 06 14:23:46.601396 2017] [:error] [pid 12985] [client 10.179.100.156:54027] PHP Warning: array_filter() expects parameter 1 to be array, null given in /var/www/html/fog/lib/fog/image.class.php on line 164, referer: http://fogserver/fog/management/index.php?node=image&sub=membership&id=24
      [Wed Sep 06 15:01:43.531662 2017] [:error] [pid 12985] [client 10.179.100.176:57724] PHP Warning: ftp_chmod(): SITE CHMOD command failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 137
      [Wed Sep 06 15:44:05.209565 2017] [:error] [pid 188731] [client 192.168.100.171:49720] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Wed Sep 06 16:07:37.877958 2017] [:error] [pid 12985] [client 192.168.100.171:49720] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Malwarebytes.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Wed Sep 06 16:10:13.754976 2017] [:error] [pid 12985] [client 192.168.100.171:49749] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Office_v2013_PP.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Wed Sep 06 17:23:14.101811 2017] [:error] [pid 158225] [client 192.168.100.171:51205] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Wed Sep 06 18:06:31.179787 2017] [:error] [pid 8849] [client 192.168.100.171:49767] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe😞 failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
      [Wed Sep 06 18:29:18.857955 2017] [:error] [pid 243543] [client 10.179.100.156:56865] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823, referer: http://fogserver/fog/management/index.php?node=snapin&sub=edit&id=42
      [root@Sal1FOGV1 bin]#

      Jim

      Wayne WorkmanW 1 Reply Last reply Reply Quote 0
      • Wayne WorkmanW
        Wayne Workman @Jim Graczyk
        last edited by

        @jim-graczyk lots of FTP errors. I’d say check your FTP credentials to be sure they are correct. There’s a test here you can do: https://wiki.fogproject.org/wiki/index.php?title=Troubleshoot_FTP#Try_to_get_a_file_with_Windows:

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
        Daily Clean Installation Results:
        https://fogtesting.fogproject.us/
        FOG Reporting:
        https://fog-external-reporting-results.fogproject.us/

        J 1 Reply Last reply Reply Quote 0
        • J
          Jim Graczyk @Wayne Workman
          last edited by

          @wayne-workman

          I’m thinking the FTP errors are the result of failed replication, not the cause - again, but what do I know…
          I’m thinking this only because the FTP errors are from FOG client machines attempting to access a snapin that hasn’t replicated.

          I’ll test the FTP connection of each node for files I know are actually on each node and report back…

          Jim

          J 1 Reply Last reply Reply Quote 0
          • J
            Jim Graczyk @Jim Graczyk
            last edited by

            @wayne-workman

            I was able to connect to each node’s FTP service with the credentials stored in the storage definition in FOG and transfer files into and out of each server.

            Jim

            Wayne WorkmanW 1 Reply Last reply Reply Quote 0
            • Wayne WorkmanW
              Wayne Workman @Jim Graczyk
              last edited by

              @jim-graczyk I’m looking through the recent commits to the working branch of FogProject: https://github.com/FOGProject/fogproject/commits/working

              Tom did push a comment to correct issues with replication in there. So what I’d recommend is updating to the working branch to see if it’s fixed or not.

              If you’re unfamiliar with how to switch branches and update, go to your git repo location on your servers (each server needs done). Do this first:
              git pull
              Then this:
              git checkout working
              then the usual:
              cd bin
              ./installfog.sh -y

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
              Daily Clean Installation Results:
              https://fogtesting.fogproject.us/
              FOG Reporting:
              https://fog-external-reporting-results.fogproject.us/

              J 1 Reply Last reply Reply Quote 0
              • J
                Jim Graczyk @Wayne Workman
                last edited by

                @wayne-workman All nodes are already up-to-date (as reported by git pull). We had updated to v24 at 10 am this morning and retested replication before posting this issue to the forum.

                Jim

                Wayne WorkmanW 1 Reply Last reply Reply Quote 0
                • Wayne WorkmanW
                  Wayne Workman @Jim Graczyk
                  last edited by

                  @jim-graczyk Ok then. I think you’ve found an issue with replication in RC8/working. Without waiting for the developers, you have two options. 1 is to go back to RC 7 where it worked. 2 is to just manually scp the image & snapin changes to the nodes as appropriate and see if that works.

                  Here’s how to go back to rc7:

                  git checkout 31a61db2c12ebc394ea167f9b37ba6ef4da7ea99
cd bin
./installfog.sh -y

                  Normally I don’t recommend downgrading but it looks like no DB changes have happened since then and now, so it should work in this case. (future readers, it will not work for you).

                  When our developers come back from vacation, hopefully they can resolve the issue.

                  AND - I feel it’s time to build some quality-checking for replication - so I’ll be working on that in my free time in the coming weekends so that we can immediately know when this stuff isn’t working in one of the branches.

                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                  Daily Clean Installation Results:
                  https://fogtesting.fogproject.us/
                  FOG Reporting:
                  https://fog-external-reporting-results.fogproject.us/

                  J Tom ElliottT 2 Replies Last reply Reply Quote 0
                  • J
                    Jim Graczyk @Wayne Workman
                    last edited by

                    @wayne-workman

                    I prefer to move forward rather than backward so I’ll choose Option B - manually replicate and test that it’s only a replication problem. We’re already working on that. This will work for a few days, but we’ll have problems as soon as we have to upload PCs before imaging.

                    I’ll post here if all images and snapins work to clients at each site.

                    Jim

                    1 Reply Last reply Reply Quote 0
                    • Tom ElliottT
                      Tom Elliott @Wayne Workman
                      last edited by

                      @wayne-workman while quality assurance is always a good thing, the code I added would not have broken what is being reported here. I added a simple check to find out if it can reach the server on port 21, the ftp port. If it cannot communicate it will report it cannot. If it does communicate it will perform replication tasks.

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                      J 1 Reply Last reply Reply Quote 0
                      • J
                        Jim Graczyk @Tom Elliott
                        last edited by

                        @tom-elliott

                        thanks Tom… we’ve tested FTP and found it working on all nodes…

                        Jim

                        1 Reply Last reply Reply Quote 0
                        • S
                          Sebastian Roth Moderator
                          last edited by

                          @Moderators @Testers Is anyone able to replicate this issue?

                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                          Wayne WorkmanW 1 Reply Last reply Reply Quote 0
                          • Wayne WorkmanW
                            Wayne Workman @Sebastian Roth
                            last edited by

                            @sebastian-roth I have plans (in my head) to build automated functional testing for this, I’m not setup to test replication at the moment.

                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                            Daily Clean Installation Results:
                            https://fogtesting.fogproject.us/
                            FOG Reporting:
                            https://fog-external-reporting-results.fogproject.us/

                            1 Reply Last reply Reply Quote 0
                            • J
                              Jim Graczyk
                              last edited by

                              I just wanted to let the FOG team know that I have just completed the build of a new FOG server using v1.5.0 RC9, Dev Branch, with associated storage nodes. We created storage groups and storage nodes, after uploading images. Installed the Locations Plugin and set locations. We moved our images and snapins to the new FOG installation…

                              And we’ve reproduced the same problem we’ve had on FOG system on which this original posting was based. We have no replication of images or snapin, even though the storage nodes are working as expected.

                              We’ve written a bash script to replicate both snapins and images from the main fog server to the storage nodes, so our installation works, but based on my experience, it’s very easy to reproduce this replications problem - just do an installation on fresh servera and the problems ensue.

                              Thanks,

                              Jim

                              Tom ElliottT 1 Reply Last reply Reply Quote 0
                              • Tom ElliottT
                                Tom Elliott @Jim Graczyk
                                last edited by

                                @jim-graczyk Can you try on working branch please? There was an issue with implementation of finding “isAvailable” nodes which I’m pretty sure has been corrected for.

                                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                J 1 Reply Last reply Reply Quote 0
                                • J
                                  Jim Graczyk @Tom Elliott
                                  last edited by

                                  @tom-elliott

                                  Tom,

                                  I’m using the working branch on my lab set up. I’ll pull it there.

                                  I’m also OK trying the working branch on my new installation, as long as I can switch back to the Dev branch after pulling the current Working version.

                                  Is it just a matter of changing the git checkout back to dev branch?

                                  FYI - My new installation is showing SVN 6079 while the lab is showing SVN 6080 - if that helps any.

                                  thanks,

                                  Jim

                                  Tom ElliottT 1 Reply Last reply Reply Quote 0
                                  • Tom ElliottT
                                    Tom Elliott @Jim Graczyk
                                    last edited by

                                    @jim-graczyk Switching to dev-branch is as easy as git checkout dev-branch (re-run installer of course too).

                                    However, reinstalling would put it back into state where it isn’t working properly again. I don’t expect the current working to remain working for too much longer though. Probably by end of this week I’ll make it into RC-10.

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                    J 1 Reply Last reply Reply Quote 0
                                    • J
                                      Jim Graczyk @Tom Elliott
                                      last edited by

                                      @tom-elliott

                                      Updated Lab to v52. Can no longer check replications logs.

                                      Fog Configuration / Log Viewer gives me a Files: pulldown that is now empty.

                                      Jim

                                      J 1 Reply Last reply Reply Quote 0
                                      • J
                                        Jim Graczyk @Jim Graczyk
                                        last edited by

                                        0_1506455177528_d68906d4-e947-4a64-a330-f2ff89017587-image.png

                                        Tom ElliottT 1 Reply Last reply Reply Quote 0
                                        • Tom ElliottT
                                          Tom Elliott @Jim Graczyk
                                          last edited by

                                          @jim-graczyk I’ll fix that later. Sorry that’s a missed thing in jquery call. The logs are still ‘working’, you just can’t view them currently from the GUI.

                                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                          1 Reply Last reply Reply Quote 0
                                          • S
                                            Sebastian Roth Moderator
                                            last edited by

                                            @Jim-Graczyk Try less /opt/fog/log/fogreplicator.log on the FOG server command line.

                                            Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                            Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                            1 Reply Last reply Reply Quote 1
                                            • 1
                                            • 2
                                            • 1 / 2
                                            • First post
                                              Last post

                                            191

                                            Online

                                            11.6k

                                            Users

                                            17.1k

                                            Topics

                                            154.6k

                                            Posts
                                            Copyright © 2012-2024 FOG Project