Posts made by Neil Underwood

Here’s another side-effect of whatever is ailing my server here:

All of the nodes now show (#!db) instead of whatever it’s supposed to show (version number I think?)
Could it be a corrupt database that’s causing all of this?

@Tom-Elliott said in FOGImagereplicator not working:

@Neil-Underwood Please give this a shot:

sudo -i
service FOGImageReplicator stop
service FOGSnapinReplicator stop
service FOGMulticastManager stop
service FOGPingHosts stop
service FOGScheduler stop
sleep 10
service FOGImageReplicator starts
service FOGSnapinReplicator start
service FOGMulticastManager start
service FOGPingHosts start
service FOGScheduler start

Ubuntu is realy strange. If there’s a pid file, but the process died, it will fail to allow the next item to startup (at least not properly). Restart appears to make it not remove the lock, at least in my experiences.

Also, if you’re running from a GUI 14.04, make sure when you go into sudo on the terminal, you first jump up to root with sudo -i. The normal terminal window (even if you are logged in to the Ubuntu system as root) will not source all the paths as needed. This is what I’ve seen, of course I have been wrong.

I tried this with no success. Did it twice to make sure. The first time I stopped FOGImageReplicator and FOGSnapinReplicator they both complained about no existing process, but agreed to close anyway. The second time I did it they did not complain but the outcome was the same.

Looking more closely at all my log files, I see that only servicemaster.log is current. Everything else hasn’t changed since March/April. Looking at multicast.log I see something that looks very wrong:

[04-04-16 9:08:19 am]  * No tasks found!
[04-04-16 9:08:29 am]  * No tasks found!
[04-04-16 9:08:39 am]  * No tasks found!
[04-04-16 9:08:49 am]  * No tasks found!
[04-04-16 9:08:59 am]  * No tasks found!
[04-04-16 9:09:09 am]  * No tasks found!
[04-04-16 9:09:19 am]  | This is not the master node
[04-04-16 9:09:29 am]  | This is not the master node
[04-04-16 9:09:39 am]  | This is not the master node
[04-04-16 9:09:49 am]  | This is not the master node
[04-04-16 9:09:59 am]  | This is not the master node
[04-04-16 9:10:09 am]  | This is not the master node
[04-04-16 9:10:19 am]  | This is not the master node
[04-04-16 9:10:29 am]  | This is not the master node
[04-04-16 9:10:39 am]  | This is not the master node
[04-04-16 9:10:49 am]  | This is not the master node
[04-04-16 9:10:59 am]  | This is not the master node
[04-04-16 9:11:09 am]  | This is not the master node
[04-04-16 9:11:19 am]  | This is not the master node
[04-04-16 9:11:29 am]  | This is not the master node
[04-04-16 9:11:39 am]  | This is not the master node
[04-04-16 9:11:49 am]  | This is not the master node
[04-04-16 9:11:59 am]  | This is not the master node
[04-04-16 9:12:09 am]  | This is not the master node
[04-04-16 9:12:19 am]  | This is not the master node
[04-04-16 9:12:29 am]  | This is not the master node
[04-04-16 9:12:39 am]  | This is not the master node
[04-04-16 9:12:49 am]  | This is not the master node

Not sure how this box lost master node status. It’s never been changed. That is a bit worrisome though. I feel like there are multiple issues with this setup.

I’m on Ubuntu 14.04. No firewall, no SElinux. In the web interface I get “Unable to open file for reading” for all apache logs. When viewing them manually I get:

[Mon Jun 20 07:35:03.417754 2016] [mpm_prefork:notice] [pid 1761] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2h configured -- resuming normal operations
[Mon Jun 20 07:35:03.417771 2016] [core:notice] [pid 1761] AH00094: Command line: '/usr/sbin/apache2'
[Mon Jun 20 14:39:59.444514 2016] [:error] [pid 14709] [client 192.168.18.151:38917] PHP Warning:  fopen(/var/log/apache2/error.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:03.291699 2016] [:error] [pid 14419] [client 192.168.18.151:38918] PHP Warning:  fopen(/var/log/apache2/access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:04.386138 2016] [:error] [pid 14420] [client 192.168.18.151:38921] PHP Warning:  fopen(/var/log/apache2/access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:06.568418 2016] [:error] [pid 14417] [client 192.168.18.151:38923] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:09.496948 2016] [:error] [pid 14419] [client 192.168.18.151:38924] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:13.349986 2016] [:error] [pid 14455] [client 192.168.18.151:38926] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:14.435769 2016] [:error] [pid 14418] [client 192.168.18.151:38928] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:16.612700 2016] [:error] [pid 14709] [client 192.168.18.151:38929] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:19.541410 2016] [:error] [pid 14418] [client 192.168.18.151:38930] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:23.393842 2016] [:error] [pid 14421] [client 192.168.18.151:38931] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:24.482583 2016] [:error] [pid 14709] [client 192.168.18.151:38933] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:26.658868 2016] [:error] [pid 14456] [client 192.168.18.151:38934] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:29.587489 2016] [:error] [pid 14455] [client 192.168.18.151:38935] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:33.436966 2016] [:error] [pid 14709] [client 192.168.18.151:38936] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:34.529403 2016] [:error] [pid 14418] [client 192.168.18.151:38938] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:36.700967 2016] [:error] [pid 14419] [client 192.168.18.151:38939] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:39.632103 2016] [:error] [pid 17625] [client 192.168.18.151:38940] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:43.481757 2016] [:error] [pid 14456] [client 192.168.18.151:38941] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:44.561740 2016] [:error] [pid 14527] [client 192.168.18.151:38943] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:46.744553 2016] [:error] [pid 14420] [client 192.168.18.151:38944] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:49.667644 2016] [:error] [pid 14456] [client 192.168.18.151:38945] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:53.536456 2016] [:error] [pid 17625] [client 192.168.18.151:38948] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:54.604695 2016] [:error] [pid 14709] [client 192.168.18.151:38950] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:56.789575 2016] [:error] [pid 14527] [client 192.168.18.151:38951] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:40:59.708623 2016] [:error] [pid 14527] [client 192.168.18.151:38952] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:41:03.581499 2016] [:error] [pid 14419] [client 192.168.18.151:38955] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:41:04.648658 2016] [:error] [pid 14527] [client 192.168.18.151:38956] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:41:06.832612 2016] [:error] [pid 17625] [client 192.168.18.151:38957] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:41:09.749289 2016] [:error] [pid 14420] [client 192.168.18.151:38958] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:41:13.625475 2016] [:error] [pid 14456] [client 192.168.18.151:38960] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer
[Mon Jun 20 14:41:14.692155 2016] [:error] [pid 17625] [client 192.168.18.151:38961] PHP Warning:  fopen(/var/log/apache2/other_vhosts_access.log): failed to open stream: Permission denied in /var/www/fog/status/logtoview.php on line 14, referer: http://192.168.19.200/fog/management/index.php?node=about&sub=logviewer

The fogreplicator.log file is completely empty. I deleted it and recreated it a few weeks ago during my troubleshooting. Prior to that it contained entries up to March and then it stopped.

Permissions in /var/log/apache2 are:

sradmin@new-fog-server ~ $ ll /var/log/apache2/*
-rw-r----- 1 root adm        0 Jun 17 07:35 /var/log/apache2/access.log
-rw-r----- 1 root adm      414 Jun 16 15:45 /var/log/apache2/access.log.1
-rwxr-xr-x 1 root adm      147 May  9 06:01 /var/log/apache2/access.log.10.gz
-rwxr-xr-x 1 root adm      167 May  7 07:55 /var/log/apache2/access.log.11.gz
-rwxr-xr-x 1 root adm      718 May  7 02:15 /var/log/apache2/access.log.12.gz
-rwxr-xr-x 1 root adm      287 May  6 07:04 /var/log/apache2/access.log.13.gz
-rwxr-xr-x 1 root adm      336 May  5 06:05 /var/log/apache2/access.log.14.gz
-rw-r----- 1 root adm      165 Jun 15 11:16 /var/log/apache2/access.log.2.gz
-rw-r----- 1 root adm      210 Jun 13 15:42 /var/log/apache2/access.log.3.gz
-rw-r----- 1 root adm      101 Jun 10 08:03 /var/log/apache2/access.log.4.gz
-rwxr-xr-x 1 root adm      252 Jun  9 22:58 /var/log/apache2/access.log.5.gz
-rwxr-xr-x 1 root adm      423 Jun  9 11:27 /var/log/apache2/access.log.6.gz
-rwxr-xr-x 1 root adm      171 May 25 15:30 /var/log/apache2/access.log.7.gz
-rwxr-xr-x 1 root adm      147 May 23 02:06 /var/log/apache2/access.log.8.gz
-rwxr-xr-x 1 root adm      147 May 17 14:23 /var/log/apache2/access.log.9.gz
-rw-r----- 1 root adm    74862 Jun 20 14:49 /var/log/apache2/error.log
-rw-r----- 1 root adm      373 Jun 20 07:35 /var/log/apache2/error.log.1
-rw-r----- 1 root adm      252 Jun 11 07:35 /var/log/apache2/error.log.10.gz
-rwxr-xr-x 1 root adm     2720 Jun 10 06:34 /var/log/apache2/error.log.11.gz
-rwxr-xr-x 1 root adm    62439 Jun  9 11:32 /var/log/apache2/error.log.12.gz
-rwxr-xr-x 1 root adm      249 May 31 07:35 /var/log/apache2/error.log.13.gz
-rwxr-xr-x 1 root adm      249 May 30 07:35 /var/log/apache2/error.log.14.gz
-rw-r----- 1 root adm      247 Jun 19 07:35 /var/log/apache2/error.log.2.gz
-rw-r----- 1 root adm      250 Jun 18 07:35 /var/log/apache2/error.log.3.gz
-rw-r----- 1 root adm      272 Jun 17 07:35 /var/log/apache2/error.log.4.gz
-rw-r----- 1 root adm      271 Jun 16 07:35 /var/log/apache2/error.log.5.gz
-rw-r----- 1 root adm      702 Jun 15 07:35 /var/log/apache2/error.log.6.gz
-rw-r----- 1 root adm      463 Jun 14 07:35 /var/log/apache2/error.log.7.gz
-rw-r----- 1 root adm      247 Jun 13 07:35 /var/log/apache2/error.log.8.gz
-rw-r----- 1 root adm      246 Jun 12 07:35 /var/log/apache2/error.log.9.gz
-rw-r----- 1 root adm 15458893 Jun 20 14:49 /var/log/apache2/other_vhosts_access.log
-rw-r----- 1 root adm 37466046 Jun 20 07:35 /var/log/apache2/other_vhosts_access.log.1
-rw-r----- 1 root adm  1439995 Jun 11 07:35 /var/log/apache2/other_vhosts_access.log.10.gz
-rwxr-xr-x 1 root adm   990971 Jun 10 06:34 /var/log/apache2/other_vhosts_access.log.11.gz
-rwxr-xr-x 1 root adm 30755510 Jun  9 11:32 /var/log/apache2/other_vhosts_access.log.12.gz
-rwxr-xr-x 1 root adm  3834221 May 31 07:35 /var/log/apache2/other_vhosts_access.log.13.gz
-rwxr-xr-x 1 root adm  3817897 May 30 07:35 /var/log/apache2/other_vhosts_access.log.14.gz
-rw-r----- 1 root adm  1923689 Jun 19 07:35 /var/log/apache2/other_vhosts_access.log.2.gz
-rw-r----- 1 root adm  2099730 Jun 18 07:35 /var/log/apache2/other_vhosts_access.log.3.gz
-rw-r----- 1 root adm  2117286 Jun 17 07:35 /var/log/apache2/other_vhosts_access.log.4.gz
-rw-r----- 1 root adm  1921642 Jun 16 07:35 /var/log/apache2/other_vhosts_access.log.5.gz
-rw-r----- 1 root adm  1904000 Jun 15 07:35 /var/log/apache2/other_vhosts_access.log.6.gz
-rw-r----- 1 root adm  1886950 Jun 14 07:35 /var/log/apache2/other_vhosts_access.log.7.gz
-rw-r----- 1 root adm  1449643 Jun 13 07:35 /var/log/apache2/other_vhosts_access.log.8.gz
-rw-r----- 1 root adm  1442578 Jun 12 07:35 /var/log/apache2/other_vhosts_access.log.9.gz

sradmin@new-fog-server ~ $ ll /opt/fog/log/*
-rwxrwxrwx 1 fog root       0 Jun  6 11:43 /opt/fog/log/fogreplicator.log
-rwxrwxrwx 1 fog root 3500683 Apr  4 09:28 /opt/fog/log/fogscheduler.log
-rwxrwxrwx 1 fog root  964464 Mar  4 12:03 /opt/fog/log/fogsnapinrep.log
-rwxrwxrwx 1 fog root  355229 Mar  9 01:39 /opt/fog/log/groupmanager.log
-rwxrwxrwx 1 fog root 8503470 Apr  4 09:29 /opt/fog/log/multicast.log
-rwxrwxrwx 1 fog root     139 Aug 25  2015 /opt/fog/log/multicast.log.udpcast.4
-rwxrwxrwx 1 fog root   95930 Aug 25  2015 /opt/fog/log/multicast.log.udpcast.5
-rwxrwxrwx 1 fog root  346282 Apr  4 09:27 /opt/fog/log/pinghosts.log
-rwxrwxrwx 1 fog root 7270442 Jun 20 14:49 /opt/fog/log/servicemaster.log

sradmin@new-fog-server ~ $ ll /var/log/fog/*
-rwxrwxrwx 1 fog root       0 Jun  6 11:43 /var/log/fog/fogreplicator.log
-rwxrwxrwx 1 fog root 3500683 Apr  4 09:28 /var/log/fog/fogscheduler.log
-rwxrwxrwx 1 fog root  964464 Mar  4 12:03 /var/log/fog/fogsnapinrep.log
-rwxrwxrwx 1 fog root  355229 Mar  9 01:39 /var/log/fog/groupmanager.log
-rwxrwxrwx 1 fog root 8503470 Apr  4 09:29 /var/log/fog/multicast.log
-rwxrwxrwx 1 fog root     139 Aug 25  2015 /var/log/fog/multicast.log.udpcast.4
-rwxrwxrwx 1 fog root   95930 Aug 25  2015 /var/log/fog/multicast.log.udpcast.5
-rwxrwxrwx 1 fog root  346282 Apr  4 09:27 /var/log/fog/pinghosts.log
-rwxrwxrwx 1 fog root 7270442 Jun 20 14:49 /var/log/fog/servicemaster.log

/opt/fog/log/* permissions are still 777 from my previous troubleshooting attempts, as are /var/log/fog/*

FTP testing checks out with no issues. I can put, get and delete just fine.

Well, I have 9 nodes actually, plus the master storage node. None of them are replicating at all. They all give me the same results.

Not sure if it’s related or worth mentioning, but at the same time this started happening, I started having issues with bzImage reaching 99% and then just sitting there when trying to run a task.

I haven’t finished trying everything in the link you provided, but I did notice that all my passwords are correct. I also noticed that I can reach each node via ftp://xxx.xxx.xxx.xxx in a web browser using the credentials in the storage settings. Question though: Shouldn’t I be able to go straight to the images directory by entering:
ftp://xxx.xxx.xxx.xxx/images
into the web browser? I have to manually navigate up two levels to get to my root directory , then choose the images folder. Only then do I see the images.

Initially the default ftp directory is empty. The first time I click to navigate up one directory I end up in /home. The second time it takes me to / as expected. I believe it’s starting me out in /home/fog, because that’s the only empty directory in /home. Is this normal?

The strangest thing is I have no problems on the web interface with the network graph and the disk space usage.

For some time now my images have not been replicating and I’ve had to transfer all new images manually via scp. I’m not sure at what point this happened, but it was after an update. Additionally, there is absolutely nothing in the image replicator log. The logging stopped at some point back in March. All the other logs seem to be working fine though. How would I go about troubleshooting this?

This installation started out on version 1.2.0 over a year ago and I have been periodically upgrading through git since. At no point have I done a fresh installation. Should I have done that at some point? And, if I do, should I be able to migrate the database over fairly easily between versions?

So when trying to run your new backup script I’m running into an error right off the bat. I’m assuming it’s because of a version mismatch. the file “…/…/lib/common/utils.sh” does not exist on my system. I’m on version 7084.

Wow, it looks like you just rewrote the entire thing. Thanks for taking the time to look into this!

@Tom-Elliott Ah, thanks for that. I’ll go grab it and check it out.

OK so it’s working for me now. I lied about getting back to work, lol. I forgot about the no_root_squash on my NFS export. My new export looks like this:

(rw,insecure,sync,no_subtree_check,no_root_squash)

Additionally, I removed the trailing directories on lines 58, 62, and 65, so they now look like this:

[[ -d $IMAGEDIR/ ]] && cp -auv $IMAGEDIR/ $backupdir/ || false
errorStat $?
echo "Done"
dots " * Backing up snapins"
[[ -d $SNAPINDIR/ ]] && cp -auv $SNAPINDIR/ $backupdir/ || false
errorStat $?
dots " * Backup up reports"
[[ -d $REPORTDIR/ ]] && cp -auv $REPORTDIR/ $backupdir/ || false

This eliminated the duplicate directory in the folder structure. I think it’s back to normal now. Not sure how or why it changed.
Hope this was helpful.

Hmmmm… So I got it working, kinda. I just removed the single quotes and escape characters on line 54:

mysqldump -h"$MYSQL_HOST" -u"$MYSQL_USER" -p"$MYSQL_PASSWORD" --allow-keywords -f $MYSQL_DATABASE > $backupdir/mysql/fog.sql

That got me past the initial hurdle. The problem now seems to be NFS related (tried with -o nfsvers=3 as well). the cp -au command is reporting:

 *  * Backing up images........................................cp: failed to preserve ownership for ‘/mnt/fog_backup/images/images/lost+found’: Operation not permitted
cp: failed to preserve ownership for ‘/mnt/fog_backup/images/images/Optiplex301064bit/d1.mbr’: Operation not permitted
cp: failed to preserve ownership for ‘/mnt/fog_backup/images/images/Optiplex301064bit/d1p1.img’: Operation not permitted

Plus, now that I’m looking at it, it’s injecting an extra “images” in the folder structure. ( I think I see why it’s doing this in lines 58, 62, and 65 )
Anyway, that’s where I’m at. I need to get back to work, so I’ll revisit it in a bit. Maybe this helps

LOL sorry to be the oddball! This script works great in a cron job for a quick & easy backup!

I’m trying to run the FOGBackup.sh script, which has worked for me countless times in the past, but it’s now giving me a weird error:

Backing up MySQL database................................mysqldump: Got error: 2005: Unknown MySQL server host '\'127.0.0.1\'' (0) when trying to connect
Failed!

I’ve tried replacing the IP with that of my server, but it’s not working. Restarted mysql, no joy. It looks like it may be passing the slashes and single quotes? I dunno. If anyone has any ideas I’m all ears. Thanks.

Running Ubuntu 14.04

@Tom-Elliott I’ve only got 2 different models at the office here. Dell Optiplex 3010 and 3020. It’s happening on both of those. If you would like me to verify other systems I will have to go to some of my other sites and check (which is not a problem, I just have to wait until someone else gets here).

@Tom-Elliott said:

Just looked at the source code and I’m leaning more towards a problem with RAM on that particular system.

Is this occurring on EVERY system, or just this particular system?

I’ve checked 3 different PC’s and they are all doing it. None of them are booting via efi.

One of them happens to be the machine I use to master one of my images. I can’t put my finger on which one, but this started happening after a recent update. Right now I’m on 4451 via github.

Sorry. Posted this on my way out the door after an hour of OT trying to figure it out. It’s CentOS 7. I checked selinux and it’s disabled. I can’t check anything else right now but I have some output from /var/log/messages:

Aug 16 04:48:48 new-fog-server FOGImageReplicator: PHP Warning:  stream_set_blocking() expects exactly 2 parameters, 1 given in /var/www/html/fog/lib/fog/FOGService.class.php on line 217
Aug 16 04:48:49 new-fog-server FOGImageReplicator: PHP Warning:  stream_set_blocking() expects exactly 2 parameters, 1 given in /var/www/html/fog/lib/fog/FOGService.class.php on line 217
Aug 16 04:48:49 new-fog-server FOGImageReplicator: PHP Warning:  stream_set_blocking() expects exactly 2 parameters, 1 given in /var/www/html/fog/lib/fog/FOGService.class.php on line 217

Lots more but had to truncate.

I’m baffled. Everything was working fine and then all of a sudden this starts happening. See photos:

Can anyone assist? I’m about to nuke it and move back to Ubuntu 14.04. This CentOS box is driving me crazy.

I did find a strange issue with the tftp password in FOG settings. It somehow had been reverted to the previous password I had used. I caught this only because I got an error when trying to re-download and install the FOG kernel. Once I fixed the password the kernel downloaded and installed just fine. I just have no clue where to begin with this though.

OK. That’s what I was afraid of. No big deal and life goes on. Strange that it didn’t take the settings though. Thanks

How in the heck do I get an image to stop the download process once it’s started? I just manually created a new host for a remote machine about 20+ miles from my current location, and it apparently did not save the location that I specified when creating the host in the web-ui. This caused it to start pulling from a node nowhere near it (down the street from me, actually). I cancelled the task in the web-ui but the image is still being transferred at about 450k/s with an ETA of 21 hours. I need to stop this process and restart it so that it pulls from the local node. I really don’t want to have to get in the truck and drive all the way out there right now. Please help. Thanks!