• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

FOGImagereplicator not working

Scheduled Pinned Locked Moved
Linux Problems
3
22
7.0k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • N
    Neil Underwood
    last edited by Jun 20, 2016, 6:38 PM

    For some time now my images have not been replicating and I’ve had to transfer all new images manually via scp. I’m not sure at what point this happened, but it was after an update. Additionally, there is absolutely nothing in the image replicator log. The logging stopped at some point back in March. All the other logs seem to be working fine though. How would I go about troubleshooting this?

    This installation started out on version 1.2.0 over a year ago and I have been periodically upgrading through git since. At no point have I done a fresh installation. Should I have done that at some point? And, if I do, should I be able to migrate the database over fairly easily between versions?

    W 1 Reply Last reply Jun 20, 2016, 6:47 PM Reply Quote 0
    • W
      Wayne Workman @Neil Underwood
      last edited by Jun 20, 2016, 6:47 PM

      @Neil-Underwood You should not need to do a re-install, we can probably fix it.

      This is 99% of the time a FTP issue, 95% of the time a credentials issue, and 50% of the time caused by your web-browser’s auto-fill.

      This is a troubleshooting guide for all of that:
      https://wiki.fogproject.org/wiki/index.php?title=Troubleshoot_FTP

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
      Daily Clean Installation Results:
      https://fogtesting.fogproject.us/
      FOG Reporting:
      https://fog-external-reporting-results.fogproject.us/

      1 Reply Last reply Reply Quote 0
      • N
        Neil Underwood
        last edited by Neil Underwood Jun 20, 2016, 1:48 PM Jun 20, 2016, 7:47 PM

        I haven’t finished trying everything in the link you provided, but I did notice that all my passwords are correct. I also noticed that I can reach each node via ftp://xxx.xxx.xxx.xxx in a web browser using the credentials in the storage settings. Question though: Shouldn’t I be able to go straight to the images directory by entering:
        ftp://xxx.xxx.xxx.xxx/images
        into the web browser? I have to manually navigate up two levels to get to my root directory , then choose the images folder. Only then do I see the images.

        Initially the default ftp directory is empty. The first time I click to navigate up one directory I end up in /home. The second time it takes me to / as expected. I believe it’s starting me out in /home/fog, because that’s the only empty directory in /home. Is this normal?

        The strangest thing is I have no problems on the web interface with the network graph and the disk space usage.

        W 1 Reply Last reply Jun 20, 2016, 7:53 PM Reply Quote 0
        • W
          Wayne Workman @Neil Underwood
          last edited by Wayne Workman Jun 20, 2016, 1:54 PM Jun 20, 2016, 7:53 PM

          @Neil-Underwood The true test is to connect via CLI, as the tests show in that article, using the credentials you have in storage management.

          Also, this is a thread about replication, so you have two or mode nodes. You’re trying to connect to the problematic one correct?

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
          Daily Clean Installation Results:
          https://fogtesting.fogproject.us/
          FOG Reporting:
          https://fog-external-reporting-results.fogproject.us/

          1 Reply Last reply Reply Quote 0
          • N
            Neil Underwood
            last edited by Neil Underwood Jun 20, 2016, 2:49 PM Jun 20, 2016, 8:47 PM

            Well, I have 9 nodes actually, plus the master storage node. None of them are replicating at all. They all give me the same results.

            Not sure if it’s related or worth mentioning, but at the same time this started happening, I started having issues with bzImage reaching 99% and then just sitting there when trying to run a task.

            1 Reply Last reply Reply Quote 0
            • N
              Neil Underwood
              last edited by Jun 20, 2016, 9:15 PM

              FTP testing checks out with no issues. I can put, get and delete just fine.

              W 1 Reply Last reply Jun 20, 2016, 9:28 PM Reply Quote 0
              • W
                Wayne Workman @Neil Underwood
                last edited by Wayne Workman Jun 20, 2016, 3:30 PM Jun 20, 2016, 9:28 PM

                @Neil-Underwood I think this is a problem with the central server. Not sure exactly yet, we need some more information. What OS is the main server? Have you reviewed firewall and SELinux (if applicable)?

                For troubleshooting, please just turn off all firewalls on the nodes and main server.

                Also, inside of Web Interface -> FOG Configuration -> Log Viewer -> Apache Error Logs What do you see in there, if anything? If that’s blank, get them manually from the OS. Depending on the OS, they are in different spots. RHEL and variants are /var/log/httpd/error_log and Debian variants are /var/log/apache2/error_log

                And, you can find replication logs on the main server here: /opt/fog/log

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                Daily Clean Installation Results:
                https://fogtesting.fogproject.us/
                FOG Reporting:
                https://fog-external-reporting-results.fogproject.us/

                1 Reply Last reply Reply Quote 0
                • N
                  Neil Underwood
                  last edited by Neil Underwood Jun 20, 2016, 3:57 PM Jun 20, 2016, 9:52 PM

                  I’m on Ubuntu 14.04. No firewall, no SElinux. In the web interface I get “Unable to open file for reading” for all apache logs. When viewing them manually I get:

                  [Mon Jun 20 07:35:03.417754 2016] [mpm_prefork:notice] [pid 1761] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2h configured -- resuming normal operations
                  [Mon Jun 20 07:35:03.417771 2016] [core:notice] [pid 1761] AH00094: Command line: '/usr/sbin/apache2'
                  [Mon Jun 20 14:39:59.444514 2016] [:error] [pid 14709] [client 192.168.18.151:38917] PHP Warning:  fopen(/var/log/apache2/error.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:03.291699 2016] [:error] [pid 14419] [client 192.168.18.151:38918] PHP Warning:  fopen(/var/log/apache2/access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:04.386138 2016] [:error] [pid 14420] [client 192.168.18.151:38921] PHP Warning:  fopen(/var/log/apache2/access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:06.568418 2016] [:error] [pid 14417] [client 192.168.18.151:38923] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:09.496948 2016] [:error] [pid 14419] [client 192.168.18.151:38924] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:13.349986 2016] [:error] [pid 14455] [client 192.168.18.151:38926] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:14.435769 2016] [:error] [pid 14418] [client 192.168.18.151:38928] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:16.612700 2016] [:error] [pid 14709] [client 192.168.18.151:38929] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:19.541410 2016] [:error] [pid 14418] [client 192.168.18.151:38930] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:23.393842 2016] [:error] [pid 14421] [client 192.168.18.151:38931] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:24.482583 2016] [:error] [pid 14709] [client 192.168.18.151:38933] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:26.658868 2016] [:error] [pid 14456] [client 192.168.18.151:38934] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:29.587489 2016] [:error] [pid 14455] [client 192.168.18.151:38935] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:33.436966 2016] [:error] [pid 14709] [client 192.168.18.151:38936] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:34.529403 2016] [:error] [pid 14418] [client 192.168.18.151:38938] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:36.700967 2016] [:error] [pid 14419] [client 192.168.18.151:38939] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:39.632103 2016] [:error] [pid 17625] [client 192.168.18.151:38940] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:43.481757 2016] [:error] [pid 14456] [client 192.168.18.151:38941] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:44.561740 2016] [:error] [pid 14527] [client 192.168.18.151:38943] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:46.744553 2016] [:error] [pid 14420] [client 192.168.18.151:38944] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:49.667644 2016] [:error] [pid 14456] [client 192.168.18.151:38945] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:53.536456 2016] [:error] [pid 17625] [client 192.168.18.151:38948] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:54.604695 2016] [:error] [pid 14709] [client 192.168.18.151:38950] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:56.789575 2016] [:error] [pid 14527] [client 192.168.18.151:38951] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:40:59.708623 2016] [:error] [pid 14527] [client 192.168.18.151:38952] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:41:03.581499 2016] [:error] [pid 14419] [client 192.168.18.151:38955] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:41:04.648658 2016] [:error] [pid 14527] [client 192.168.18.151:38956] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:41:06.832612 2016] [:error] [pid 17625] [client 192.168.18.151:38957] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:41:09.749289 2016] [:error] [pid 14420] [client 192.168.18.151:38958] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:41:13.625475 2016] [:error] [pid 14456] [client 192.168.18.151:38960] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  [Mon Jun 20 14:41:14.692155 2016] [:error] [pid 17625] [client 192.168.18.151:38961] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
                  

                  The fogreplicator.log file is completely empty. I deleted it and recreated it a few weeks ago during my troubleshooting. Prior to that it contained entries up to March and then it stopped.

                  Permissions in /var/log/apache2 are:

                  sradmin@new-fog-server ~ $ ll /var/log/apache2/*
                  -rw-r----- 1 root adm        0 Jun 17 07:35 /var/log/apache2/access.log
                  -rw-r----- 1 root adm      414 Jun 16 15:45 /var/log/apache2/access.log.1
                  -rwxr-xr-x 1 root adm      147 May  9 06:01 /var/log/apache2/access.log.10.gz
                  -rwxr-xr-x 1 root adm      167 May  7 07:55 /var/log/apache2/access.log.11.gz
                  -rwxr-xr-x 1 root adm      718 May  7 02:15 /var/log/apache2/access.log.12.gz
                  -rwxr-xr-x 1 root adm      287 May  6 07:04 /var/log/apache2/access.log.13.gz
                  -rwxr-xr-x 1 root adm      336 May  5 06:05 /var/log/apache2/access.log.14.gz
                  -rw-r----- 1 root adm      165 Jun 15 11:16 /var/log/apache2/access.log.2.gz
                  -rw-r----- 1 root adm      210 Jun 13 15:42 /var/log/apache2/access.log.3.gz
                  -rw-r----- 1 root adm      101 Jun 10 08:03 /var/log/apache2/access.log.4.gz
                  -rwxr-xr-x 1 root adm      252 Jun  9 22:58 /var/log/apache2/access.log.5.gz
                  -rwxr-xr-x 1 root adm      423 Jun  9 11:27 /var/log/apache2/access.log.6.gz
                  -rwxr-xr-x 1 root adm      171 May 25 15:30 /var/log/apache2/access.log.7.gz
                  -rwxr-xr-x 1 root adm      147 May 23 02:06 /var/log/apache2/access.log.8.gz
                  -rwxr-xr-x 1 root adm      147 May 17 14:23 /var/log/apache2/access.log.9.gz
                  -rw-r----- 1 root adm    74862 Jun 20 14:49 /var/log/apache2/error.log
                  -rw-r----- 1 root adm      373 Jun 20 07:35 /var/log/apache2/error.log.1
                  -rw-r----- 1 root adm      252 Jun 11 07:35 /var/log/apache2/error.log.10.gz
                  -rwxr-xr-x 1 root adm     2720 Jun 10 06:34 /var/log/apache2/error.log.11.gz
                  -rwxr-xr-x 1 root adm    62439 Jun  9 11:32 /var/log/apache2/error.log.12.gz
                  -rwxr-xr-x 1 root adm      249 May 31 07:35 /var/log/apache2/error.log.13.gz
                  -rwxr-xr-x 1 root adm      249 May 30 07:35 /var/log/apache2/error.log.14.gz
                  -rw-r----- 1 root adm      247 Jun 19 07:35 /var/log/apache2/error.log.2.gz
                  -rw-r----- 1 root adm      250 Jun 18 07:35 /var/log/apache2/error.log.3.gz
                  -rw-r----- 1 root adm      272 Jun 17 07:35 /var/log/apache2/error.log.4.gz
                  -rw-r----- 1 root adm      271 Jun 16 07:35 /var/log/apache2/error.log.5.gz
                  -rw-r----- 1 root adm      702 Jun 15 07:35 /var/log/apache2/error.log.6.gz
                  -rw-r----- 1 root adm      463 Jun 14 07:35 /var/log/apache2/error.log.7.gz
                  -rw-r----- 1 root adm      247 Jun 13 07:35 /var/log/apache2/error.log.8.gz
                  -rw-r----- 1 root adm      246 Jun 12 07:35 /var/log/apache2/error.log.9.gz
                  -rw-r----- 1 root adm 15458893 Jun 20 14:49 /var/log/apache2/other_vhosts_access.log
                  -rw-r----- 1 root adm 37466046 Jun 20 07:35 /var/log/apache2/other_vhosts_access.log.1
                  -rw-r----- 1 root adm  1439995 Jun 11 07:35 /var/log/apache2/other_vhosts_access.log.10.gz
                  -rwxr-xr-x 1 root adm   990971 Jun 10 06:34 /var/log/apache2/other_vhosts_access.log.11.gz
                  -rwxr-xr-x 1 root adm 30755510 Jun  9 11:32 /var/log/apache2/other_vhosts_access.log.12.gz
                  -rwxr-xr-x 1 root adm  3834221 May 31 07:35 /var/log/apache2/other_vhosts_access.log.13.gz
                  -rwxr-xr-x 1 root adm  3817897 May 30 07:35 /var/log/apache2/other_vhosts_access.log.14.gz
                  -rw-r----- 1 root adm  1923689 Jun 19 07:35 /var/log/apache2/other_vhosts_access.log.2.gz
                  -rw-r----- 1 root adm  2099730 Jun 18 07:35 /var/log/apache2/other_vhosts_access.log.3.gz
                  -rw-r----- 1 root adm  2117286 Jun 17 07:35 /var/log/apache2/other_vhosts_access.log.4.gz
                  -rw-r----- 1 root adm  1921642 Jun 16 07:35 /var/log/apache2/other_vhosts_access.log.5.gz
                  -rw-r----- 1 root adm  1904000 Jun 15 07:35 /var/log/apache2/other_vhosts_access.log.6.gz
                  -rw-r----- 1 root adm  1886950 Jun 14 07:35 /var/log/apache2/other_vhosts_access.log.7.gz
                  -rw-r----- 1 root adm  1449643 Jun 13 07:35 /var/log/apache2/other_vhosts_access.log.8.gz
                  -rw-r----- 1 root adm  1442578 Jun 12 07:35 /var/log/apache2/other_vhosts_access.log.9.gz
                  
                  sradmin@new-fog-server ~ $ ll /opt/fog/log/*
                  -rwxrwxrwx 1 fog root       0 Jun  6 11:43 /opt/fog/log/fogreplicator.log
                  -rwxrwxrwx 1 fog root 3500683 Apr  4 09:28 /opt/fog/log/fogscheduler.log
                  -rwxrwxrwx 1 fog root  964464 Mar  4 12:03 /opt/fog/log/fogsnapinrep.log
                  -rwxrwxrwx 1 fog root  355229 Mar  9 01:39 /opt/fog/log/groupmanager.log
                  -rwxrwxrwx 1 fog root 8503470 Apr  4 09:29 /opt/fog/log/multicast.log
                  -rwxrwxrwx 1 fog root     139 Aug 25  2015 /opt/fog/log/multicast.log.udpcast.4
                  -rwxrwxrwx 1 fog root   95930 Aug 25  2015 /opt/fog/log/multicast.log.udpcast.5
                  -rwxrwxrwx 1 fog root  346282 Apr  4 09:27 /opt/fog/log/pinghosts.log
                  -rwxrwxrwx 1 fog root 7270442 Jun 20 14:49 /opt/fog/log/servicemaster.log
                  
                  sradmin@new-fog-server ~ $ ll /var/log/fog/*
                  -rwxrwxrwx 1 fog root       0 Jun  6 11:43 /var/log/fog/fogreplicator.log
                  -rwxrwxrwx 1 fog root 3500683 Apr  4 09:28 /var/log/fog/fogscheduler.log
                  -rwxrwxrwx 1 fog root  964464 Mar  4 12:03 /var/log/fog/fogsnapinrep.log
                  -rwxrwxrwx 1 fog root  355229 Mar  9 01:39 /var/log/fog/groupmanager.log
                  -rwxrwxrwx 1 fog root 8503470 Apr  4 09:29 /var/log/fog/multicast.log
                  -rwxrwxrwx 1 fog root     139 Aug 25  2015 /var/log/fog/multicast.log.udpcast.4
                  -rwxrwxrwx 1 fog root   95930 Aug 25  2015 /var/log/fog/multicast.log.udpcast.5
                  -rwxrwxrwx 1 fog root  346282 Apr  4 09:27 /var/log/fog/pinghosts.log
                  -rwxrwxrwx 1 fog root 7270442 Jun 20 14:49 /var/log/fog/servicemaster.log
                  

                  /opt/fog/log/* permissions are still 777 from my previous troubleshooting attempts, as are /var/log/fog/*

                  T 1 Reply Last reply Jun 21, 2016, 12:25 AM Reply Quote 0
                  • T
                    Tom Elliott @Neil Underwood
                    last edited by Jun 21, 2016, 12:25 AM

                    @Neil-Underwood Please give this a shot:

                    sudo -i
                    service FOGImageReplicator stop
                    service FOGSnapinReplicator stop
                    service FOGMulticastManager stop
                    service FOGPingHosts stop
                    service FOGScheduler stop
                    sleep 10
                    service FOGImageReplicator starts
                    service FOGSnapinReplicator start
                    service FOGMulticastManager start
                    service FOGPingHosts start
                    service FOGScheduler start
                    

                    Ubuntu is realy strange. If there’s a pid file, but the process died, it will fail to allow the next item to startup (at least not properly). Restart appears to make it not remove the lock, at least in my experiences.

                    Also, if you’re running from a GUI 14.04, make sure when you go into sudo on the terminal, you first jump up to root with sudo -i. The normal terminal window (even if you are logged in to the Ubuntu system as root) will not source all the paths as needed. This is what I’ve seen, of course I have been wrong.

                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                    W N 2 Replies Last reply Jun 21, 2016, 12:55 AM Reply Quote 1
                    • W
                      Wayne Workman @Tom Elliott
                      last edited by Jun 21, 2016, 12:55 AM

                      @Tom-Elliott said in FOGImagereplicator not working:

                      @Neil-Underwood Please give this a shot:

                      sudo -i
                      service FOGImageReplicator stop
                      service FOGSnapinReplicator stop
                      service FOGMulticastManager stop
                      service FOGPingHosts stop
                      service FOGScheduler stop
                      sleep 10
                      service FOGImageReplicator starts
                      service FOGSnapinReplicator start
                      service FOGMulticastManager start
                      service FOGPingHosts start
                      service FOGScheduler start
                      

                      Ubuntu is realy strange. If there’s a pid file, but the process died, it will fail to allow the next item to startup (at least not properly). Restart appears to make it not remove the lock, at least in my experiences.

                      Also, if you’re running from a GUI 14.04, make sure when you go into sudo on the terminal, you first jump up to root with sudo -i. The normal terminal window (even if you are logged in to the Ubuntu system as root) will not source all the paths as needed. This is what I’ve seen, of course I have been wrong.

                      #wiki worthy

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                      Daily Clean Installation Results:
                      https://fogtesting.fogproject.us/
                      FOG Reporting:
                      https://fog-external-reporting-results.fogproject.us/

                      1 Reply Last reply Reply Quote 0
                      • N
                        Neil Underwood @Tom Elliott
                        last edited by Jun 21, 2016, 1:57 PM

                        @Tom-Elliott said in FOGImagereplicator not working:

                        @Neil-Underwood Please give this a shot:

                        sudo -i
                        service FOGImageReplicator stop
                        service FOGSnapinReplicator stop
                        service FOGMulticastManager stop
                        service FOGPingHosts stop
                        service FOGScheduler stop
                        sleep 10
                        service FOGImageReplicator starts
                        service FOGSnapinReplicator start
                        service FOGMulticastManager start
                        service FOGPingHosts start
                        service FOGScheduler start
                        

                        Ubuntu is realy strange. If there’s a pid file, but the process died, it will fail to allow the next item to startup (at least not properly). Restart appears to make it not remove the lock, at least in my experiences.

                        Also, if you’re running from a GUI 14.04, make sure when you go into sudo on the terminal, you first jump up to root with sudo -i. The normal terminal window (even if you are logged in to the Ubuntu system as root) will not source all the paths as needed. This is what I’ve seen, of course I have been wrong.

                        I tried this with no success. Did it twice to make sure. The first time I stopped FOGImageReplicator and FOGSnapinReplicator they both complained about no existing process, but agreed to close anyway. The second time I did it they did not complain but the outcome was the same.

                        Looking more closely at all my log files, I see that only servicemaster.log is current. Everything else hasn’t changed since March/April. Looking at multicast.log I see something that looks very wrong:

                        [04-04-16 9:08:19 am]  * No tasks found!
                        [04-04-16 9:08:29 am]  * No tasks found!
                        [04-04-16 9:08:39 am]  * No tasks found!
                        [04-04-16 9:08:49 am]  * No tasks found!
                        [04-04-16 9:08:59 am]  * No tasks found!
                        [04-04-16 9:09:09 am]  * No tasks found!
                        [04-04-16 9:09:19 am]  | This is not the master node
                        [04-04-16 9:09:29 am]  | This is not the master node
                        [04-04-16 9:09:39 am]  | This is not the master node
                        [04-04-16 9:09:49 am]  | This is not the master node
                        [04-04-16 9:09:59 am]  | This is not the master node
                        [04-04-16 9:10:09 am]  | This is not the master node
                        [04-04-16 9:10:19 am]  | This is not the master node
                        [04-04-16 9:10:29 am]  | This is not the master node
                        [04-04-16 9:10:39 am]  | This is not the master node
                        [04-04-16 9:10:49 am]  | This is not the master node
                        [04-04-16 9:10:59 am]  | This is not the master node
                        [04-04-16 9:11:09 am]  | This is not the master node
                        [04-04-16 9:11:19 am]  | This is not the master node
                        [04-04-16 9:11:29 am]  | This is not the master node
                        [04-04-16 9:11:39 am]  | This is not the master node
                        [04-04-16 9:11:49 am]  | This is not the master node
                        [04-04-16 9:11:59 am]  | This is not the master node
                        [04-04-16 9:12:09 am]  | This is not the master node
                        [04-04-16 9:12:19 am]  | This is not the master node
                        [04-04-16 9:12:29 am]  | This is not the master node
                        [04-04-16 9:12:39 am]  | This is not the master node
                        [04-04-16 9:12:49 am]  | This is not the master node
                        

                        Not sure how this box lost master node status. It’s never been changed. That is a bit worrisome though. I feel like there are multiple issues with this setup.

                        1 Reply Last reply Reply Quote 0
                        • N
                          Neil Underwood
                          last edited by Neil Underwood Jun 21, 2016, 8:11 AM Jun 21, 2016, 2:11 PM

                          Here’s another side-effect of whatever is ailing my server here:
                          0_1466518209461_Screenshot from 2016-06-21 07:08:22.png
                          All of the nodes now show (#!db) instead of whatever it’s supposed to show (version number I think?)
                          Could it be a corrupt database that’s causing all of this?

                          1 Reply Last reply Reply Quote 0
                          • W
                            Wayne Workman
                            last edited by Jun 21, 2016, 2:16 PM

                            Looking at that picture, this issue is possibly related: https://forums.fogproject.org/topic/7825/db/17

                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                            Daily Clean Installation Results:
                            https://fogtesting.fogproject.us/
                            FOG Reporting:
                            https://fog-external-reporting-results.fogproject.us/

                            N 1 Reply Last reply Jun 21, 2016, 2:34 PM Reply Quote 0
                            • N
                              Neil Underwood @Wayne Workman
                              last edited by Jun 21, 2016, 2:34 PM

                              @Wayne-Workman said in FOGImagereplicator not working:

                              Looking at that picture, this issue is possibly related: https://forums.fogproject.org/topic/7825/db/17

                              You know, that prompted me to look at class.config.php and I think the DATABASE_HOST value is wrong, but I really don’t know. Can you verify this is correct? Should the “p:” be in there proceeding the IP address?

                              define('DATABASE_HOST','p:127.0.0.1');
                              

                              I tried changing it and removing the “p:” and nothing seemed to change (after restarting php, mysql & apache just to be sure)

                              W 1 Reply Last reply Jun 21, 2016, 2:43 PM Reply Quote 0
                              • W
                                Wayne Workman @Neil Underwood
                                last edited by Jun 21, 2016, 2:43 PM

                                @Neil-Underwood so, for the main server that hosts the DB, that is correct. For all nodes, it is wrong.

                                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                Daily Clean Installation Results:
                                https://fogtesting.fogproject.us/
                                FOG Reporting:
                                https://fog-external-reporting-results.fogproject.us/

                                1 Reply Last reply Reply Quote 0
                                • N
                                  Neil Underwood
                                  last edited by Jun 21, 2016, 2:52 PM

                                  OK, so now I’m seeing that I can’t connect to the master storage node DB from the remote storage nodes. I know as much about SQL commands and permissions as my left shoe. How do I fix this? And how in the heck did it break in the first place?

                                  W 1 Reply Last reply Jun 21, 2016, 4:03 PM Reply Quote 0
                                  • W
                                    Wayne Workman @Neil Underwood
                                    last edited by Jun 21, 2016, 4:03 PM

                                    @Neil-Underwood said in FOGImagereplicator not working:

                                    w do I fix this? And how in the heck did it break in the first place?

                                    Probably the server is only allowing local connections. There’s stuff about that in here:
                                    https://wiki.fogproject.org/wiki/index.php?title=Troubleshoot_MySQL

                                    “Enable Remote MySQL Access”

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                    Daily Clean Installation Results:
                                    https://fogtesting.fogproject.us/
                                    FOG Reporting:
                                    https://fog-external-reporting-results.fogproject.us/

                                    N 1 Reply Last reply Jun 21, 2016, 9:23 PM Reply Quote 0
                                    • N
                                      Neil Underwood @Wayne Workman
                                      last edited by Neil Underwood Jun 21, 2016, 3:25 PM Jun 21, 2016, 9:23 PM

                                      @Wayne-Workman said in FOGImagereplicator not working:

                                      @Neil-Underwood said in FOGImagereplicator not working:

                                      w do I fix this? And how in the heck did it break in the first place?

                                      Probably the server is only allowing local connections. There’s stuff about that in here:
                                      https://wiki.fogproject.org/wiki/index.php?title=Troubleshoot_MySQL

                                      “Enable Remote MySQL Access”

                                      I must have overlooked the part about remote access because I visited this page earlier this morning. I fixed the remote permissions and now everything is able to see the DB. Thanks for that. I did do an OS upgrade that probably changed mysql versions. I’m betting that’s what screwed it up.

                                      The mystery of where my log info is going still remains, however.

                                      Also, on the dashboard the replicator appears to be working, based on the graph. But running jnettop from the master storage node doesn’t show any replication activity. If I can get my logs working again I’m certain I can figure out what the problem is.

                                      W 1 Reply Last reply Jun 21, 2016, 9:27 PM Reply Quote 0
                                      • W
                                        Wayne Workman @Neil Underwood
                                        last edited by Wayne Workman Jun 21, 2016, 3:27 PM Jun 21, 2016, 9:27 PM

                                        @Neil-Underwood sometimes it takes a hot second for replication to kick in, even if the log says it’s going. Just be patient. Tomorrow morning you’ll be able to tell if everything is right or not.

                                        Also, the logs might be fixed by re-running the installer on all the nodes, since it can now actually communicate with the DB.

                                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                        Daily Clean Installation Results:
                                        https://fogtesting.fogproject.us/
                                        FOG Reporting:
                                        https://fog-external-reporting-results.fogproject.us/

                                        1 Reply Last reply Reply Quote 0
                                        • N
                                          Neil Underwood
                                          last edited by Jun 22, 2016, 5:01 PM

                                          OK so I’m updated to 8215 now and everything seems to be able to see each other. It looks like the main server is pushing our the smartinstaller.exe to all the clients and that’s the activity I’ve been seeing on the graph. Hard to catch that using jnettop at first.

                                          Thanks for all the assistance with this. I’ll open a separate thread for the log issue and another for the bzImage problem.

                                          One more quick question though. What are these values used for?

                                          startrange='192.168.19.10'
                                          endrange='192.168.19.254'
                                          
                                          T 1 Reply Last reply Jun 22, 2016, 5:03 PM Reply Quote 0
                                          • 1
                                          • 2
                                          • 1 / 2
                                          1 / 2
                                          • First post
                                            7/22
                                            Last post

                                          209

                                          Online

                                          12.0k

                                          Users

                                          17.3k

                                          Topics

                                          155.2k

                                          Posts
                                          Copyright © 2012-2024 FOG Project