build 7102 'Backing up database ... failed'

sudburr

Updating to build 7102 from 6821 has worked on 5 other servers so far, but I have one that insists on failing at …

Backing up database.........................................Failed!

From /opt/trunksvn/fogproject/bin/error_logs

Apr 07 00:01:21 xyzfog systemd[1]: Starting The PHP FastCGI Process Manager...
Apr 07 00:01:21 xyzfog systemd[1]: Started The PHP FastCGI Process Manager.
--2016-04-07 00:01:23--  http://172.19.244.32/fog//management/export.php?type=sql
Connecting to 172.19.244.32:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.
... Giving up. after 20 attempts

Also /var/log/httpd/error_log:

[Wed Apr 06 23:32:53.046956 2016] [:error] [pid 2919] [client 192.168.54.120:57605] PHP Warning:  mysqli::real_connect(): MySQL server has gone away in /var/www/html/fog/lib/db/mysql.class.php on line 29
[Wed Apr 06 23:32:57.487734 2016] [:error] [pid 2852] [client 172.22.57.44:49402] PHP Warning:  mysqli::real_connect(): MySQL server has gone away in /var/www/html/fog/lib/db/mysql.class.php on line 29
[Wed Apr 06 23:32:57.487869 2016] [:error] [pid 2852] [client 172.22.57.44:49402] PHP Warning:  mysqli::real_connect(): (HY000/2002): Connection refused in /var/www/html/fog/lib/db/mysql.class.php on line 29
[Wed Apr 06 23:32:57.487883 2016] [:error] [pid 2852] [client 172.22.57.44:49402] PHP Warning:  mysqli::set_charset(): invalid object or resource mysqli\n in /var/www/html/fog/lib/db/mysql.class.php on line 30
[Wed Apr 06 23:32:57.487896 2016] [:error] [pid 2852] [client 172.22.57.44:49402] PHP Warning:  mysqli::select_db(): invalid object or resource mysqli\n in /var/www/html/fog/lib/db/mysql.class.php on line 39
[Wed Apr 06 23:32:57.487915 2016] [:error] [pid 2852] [client 172.22.57.44:49402] PHP Warning:  mysqli::select_db(): invalid object or resource mysqli\n in /var/www/html/fog/lib/db/mysql.class.php on line 39
[Wed Apr 06 23:32:57.487927 2016] [:error] [pid 2852] [client 172.22.57.44:49402] PHP Warning:  mysqli::query(): invalid object or resource mysqli\n in /var/www/html/fog/lib/db/mysql.class.php on line 51

Each server is identically configured on CentOS7, although … The first 4 servers were originally installed onto CentOS7.1.1503, this one was originally built upon CentOS7.2.1511.

sudburr

I’m wondering now if the database backup failure was caused by the bloating of the history table like I experienced on another server? hmm …

Too bad I didn’t check for that.

sudburr

So I :

mysql -u root -p
DROP DATABASE fog;
mv /opt/fog /opt/fog.old

Then launched the installer and I’m your uncle.

sudburr

Still no love for my original problem on this particular naughty server.

Tom Elliott

@Wayne-Workman it was a switch from self to static. In the case of the base class everything needs to be self.

Wayne Workman

Whatever the issue was, it’s fixed in r7118.

Tom Elliott

try updating.

Wayne Workman

r7116 on Fedora 21 fully updated, I’m getting a db failed message as well.

* Backing up database.........................................Failed!

Here’s what’s in the error log:

/usr/bin/lsb_release
/usr/bin/systemctl
ln: failed to create symbolic link ‘/usr/lib/systemd/system/mysql.service’: File exists
ln: failed to create symbolic link ‘/usr/lib/systemd/system/mysqld.service’: File exists
ln: failed to create symbolic link ‘/etc/systemd/system/mysql.service’: File exists
ln: failed to create symbolic link ‘/etc/systemd/system/mysqld.service’: File exists
/opt/fog/.fogsettings
Loaded plugins: langpacks
http://remi.kazukioishi.net/fedora/21/remi/x86_64/repodata/repomd.xml: [Errno 12] Timeout on http://remi.kazukioishi.net/fedora/21/remi/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds')
Trying other mirror.
No package epel-release available.
Error: Nothing to do
remi-release-21-3.fc21.remi.noarch
Loaded plugins: langpacks
================================== repo: remi ==================================
[remi]
async = True
bandwidth = 0
base_persistdir = /var/lib/yum/repos/x86_64/21
baseurl =
cache = 0
cachedir = /var/cache/yum/x86_64/21/remi
check_config_file_age = True
cost = 1000
deltarpm_metadata_percentage = 100
deltarpm_percentage =
enabled = True
enablegroups = True
exclude =
failovermethod = roundrobin
gpgcadir = /var/lib/yum/repos/x86_64/21/remi/gpgcadir
gpgcakey =
gpgcheck = True
gpgdir = /var/lib/yum/repos/x86_64/21/remi/gpgdir
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-remi
hdrdir = /var/cache/yum/x86_64/21/remi/headers
http_caching = all
includepkgs =
ip_resolve =
keepalive = True
keepcache = False
mddownloadpolicy = sqlite
mdpolicy = group:small
mediaid =
metadata_expire = 21600
metadata_expire_filter = read-only:present
metalink =
minrate = 0
mirrorlist = http://rpms.remirepo.net/fedora/21/remi/mirror
mirrorlist_expire = 86400
name = Remi's RPM repository - Fedora 21 - x86_64
old_base_cache_dir =
password =
persistdir = /var/lib/yum/repos/x86_64/21/remi
pkgdir = /var/cache/yum/x86_64/21/remi/packages
proxy = False
proxy_dict =
proxy_password =
proxy_username =
repo_gpgcheck = False
retries = 10
skip_if_unavailable = False
ssl_check_cert_permissions = True
sslcacert =
sslclientcert =
sslclientkey =
sslverify = True
throttle = 0
timeout = 30.0
ui_id = remi/21
ui_repoid_vars = releasever,
   basearch
username =

Loaded plugins: langpacks
httpd-2.4.16-1.fc21.x86_64
php-5.6.20-1.fc21.remi.x86_64
php-cli-5.6.20-1.fc21.remi.x86_64
php-common-5.6.20-1.fc21.remi.x86_64
php-gd-5.6.20-1.fc21.remi.x86_64
mariadb-10.0.21-1.fc21.x86_64
mariadb-server-10.0.21-1.fc21.x86_64
tftp-server-5.2-15.fc21.x86_64
nfs-utils-1.3.1-6.4.fc21.x86_64
vsftpd-3.0.2-12.fc21.x86_64
net-tools-2.0-0.31.20141124git.fc21.x86_64
wget-1.16.3-1.fc21.x86_64
xinetd-2.3.15-15.fc21.x86_64
tar-1.27.1-8.fc21.x86_64
gzip-1.6-8.fc21.x86_64
make-4.0-3.fc21.x86_64
m4-1.4.17-6.fc21.x86_64
gcc-4.9.2-6.fc21.x86_64
gcc-c++-4.9.2-6.fc21.x86_64
lftp-4.5.4-4.fc21.x86_64
Loaded plugins: langpacks
Installed Packages
php-mysqlnd.x86_64                  5.6.20-1.fc21.remi                   @remi
Available Packages
php-mysqlnd.i686                    5.6.15-1.fc21                        updates
php-mysqlnd-5.6.20-1.fc21.remi.x86_64
curl-7.37.0-14.fc21.x86_64
php-mcrypt-5.6.20-1.fc21.remi.x86_64
php-mbstring-5.6.20-1.fc21.remi.x86_64
mod_ssl-2.4.16-1.fc21.x86_64
php-fpm-5.6.20-1.fc21.remi.x86_64
php-process-5.6.20-1.fc21.remi.x86_64
Loaded plugins: langpacks
No packages marked for update
httpd-2.4.16-1.fc21.x86_64
php-5.6.20-1.fc21.remi.x86_64
php-cli-5.6.20-1.fc21.remi.x86_64
php-common-5.6.20-1.fc21.remi.x86_64
php-gd-5.6.20-1.fc21.remi.x86_64
mariadb-10.0.21-1.fc21.x86_64
mariadb-server-10.0.21-1.fc21.x86_64
tftp-server-5.2-15.fc21.x86_64
nfs-utils-1.3.1-6.4.fc21.x86_64
vsftpd-3.0.2-12.fc21.x86_64
net-tools-2.0-0.31.20141124git.fc21.x86_64
wget-1.16.3-1.fc21.x86_64
xinetd-2.3.15-15.fc21.x86_64
tar-1.27.1-8.fc21.x86_64
gzip-1.6-8.fc21.x86_64
make-4.0-3.fc21.x86_64
m4-1.4.17-6.fc21.x86_64
gcc-4.9.2-6.fc21.x86_64
gcc-c++-4.9.2-6.fc21.x86_64
lftp-4.5.4-4.fc21.x86_64
php-mysqlnd-5.6.20-1.fc21.remi.x86_64
php-mysqlnd-5.6.20-1.fc21.remi.x86_64
curl-7.37.0-14.fc21.x86_64
php-mcrypt-5.6.20-1.fc21.remi.x86_64
php-mbstring-5.6.20-1.fc21.remi.x86_64
mod_ssl-2.4.16-1.fc21.x86_64
php-fpm-5.6.20-1.fc21.remi.x86_64
php-process-5.6.20-1.fc21.remi.x86_64
New password: Retype new password: Changing password for user fog.
passwd: all authentication tokens updated successfully.
Failed to execute operation: No such file or directory
● mariadb.service - MariaDB 10.0 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled)
  Drop-In: /usr/lib/systemd/system/mariadb.service.d
           └─mysql-compat.conf
   Active: active (running) since Fri 2016-04-08 12:56:04 CDT; 2s ago
  Process: 47341 ExecStartPost=/usr/libexec/mysql-check-upgrade (code=exited, status=0/SUCCESS)
  Process: 47189 ExecStartPost=/usr/libexec/mysql-wait-ready $MAINPID (code=exited, status=0/SUCCESS)
  Process: 47153 ExecStartPre=/usr/libexec/mysql-prepare-db-dir %n (code=exited, status=0/SUCCESS)
  Process: 47129 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
 Main PID: 47188 (mysqld_safe)
   CGroup: /system.slice/mariadb.service
           ├─47188 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
           └─47304 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/log/mariadb/mariadb.log --pid-file=/var/run/mariadb/mariadb.pid --socket=/var/lib/mysql/mysql.sock

Apr 08 12:56:03 localhost.localdomain mysqld_safe[47188]: 160408 12:56:03 mysqld_safe Logging to '/var/log/mariadb/mariadb.log'.
Apr 08 12:56:03 localhost.localdomain mysqld_safe[47188]: 160408 12:56:03 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
Signature ok
subject=/CN=10.2.1.11
Getting CA Private Key
ln: failed to create symbolic link ‘/var/www/html/fog/fog’: File exists
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled)
   Active: active (running) since Fri 2016-04-08 13:05:04 CDT; 2s ago
 Main PID: 47564 (httpd)
   Status: "Processing requests..."
   CGroup: /system.slice/httpd.service
           ├─47564 /usr/sbin/httpd -DFOREGROUND
           ├─47572 /usr/sbin/httpd -DFOREGROUND
           ├─47573 /usr/sbin/httpd -DFOREGROUND
           ├─47574 /usr/sbin/httpd -DFOREGROUND
           ├─47575 /usr/sbin/httpd -DFOREGROUND
           ├─47576 /usr/sbin/httpd -DFOREGROUND
           ├─47582 /usr/sbin/httpd -DFOREGROUND
           └─47585 /usr/sbin/httpd -DFOREGROUND

Apr 08 13:05:04 localhost.localdomain httpd[47564]: AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain. Set the 'ServerName' directive globally to suppress this message

● php-fpm.service - The PHP FastCGI Process Manager
   Loaded: loaded (/usr/lib/systemd/system/php-fpm.service; enabled)
   Active: active (running) since Fri 2016-04-08 13:05:04 CDT; 3s ago
 Main PID: 47565 (php-fpm)
   Status: "Ready to handle connections"
   CGroup: /system.slice/php-fpm.service
           ├─47565 php-fpm: master process (/etc/php-fpm.conf
           ├─47566 php-fpm: pool www
           ├─47567 php-fpm: pool www
           ├─47568 php-fpm: pool www
           ├─47569 php-fpm: pool www
           └─47570 php-fpm: pool www
--2016-04-08 13:05:08--  http://10.2.1.11/fog//management/export.php?type=sql
Connecting to 10.2.1.11:80... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2016-04-08 13:05:09 ERROR 500: Internal Server Error.

Wayne Workman

@george1421 Not sure… I’m sure it exists though.

ls /home

george1421

@Wayne-Workman what happens if the destination location does not exist?

sudburr

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       115G  3.3G  106G   4% /
devtmpfs        919M     0  919M   0% /dev
tmpfs           928M     0  928M   0% /dev/shm
tmpfs           928M  8.5M  920M   1% /run
tmpfs           928M     0  928M   0% /sys/fs/cgroup
/dev/sdb1       932G  249G  684G  27% /images
/dev/sda1       477M  172M  276M  39% /boot
tmpfs           186M     0  186M   0% /run/user/1000

Wayne Workman

@sudburr partitions, man. partitions… check anyways.

sudburr

Gigs worth of space.

Wayne Workman

@sudburr I’ve been watching this thread and I’ve not had any ideas. The database backups go into /home maybe make sure you have space there. df -h

sudburr

Command results:

mysql -u root fog -p
show tables;

+------------------------+
| Tables_in_fog          |
+------------------------+
| aloLog                 |
| clientUpdates          |
| dirCleaner             |
| globalSettings         |
| greenFog               |
| groupMembers           |
| groups                 |
| history                |
| hostAutoLogOut         |
| hostMAC                |
| hostScreenSettings     |
| hosts                  |
| imageGroupAssoc        |
| imagePartitionTypes    |
| imageTypes             |
| images                 |
| imagingLog             |
| inventory              |
| ipxeTable              |
| keySequence            |
| moduleStatusByHost     |
| modules                |
| multicastSessions      |
| multicastSessionsAssoc |
| nfsFailures            |
| nfsGroupMembers        |
| nfsGroups              |
| os                     |
| oui                    |
| plugins                |
| printerAssoc           |
| printers               |
| pxeMenu                |
| scheduledTasks         |
| schemaVersion          |
| snapinAssoc            |
| snapinGroupAssoc       |
| snapinJobs             |
| snapinTasks            |
| snapins                |
| supportedOS            |
| taskLog                |
| taskStates             |
| taskTypes              |
| tasks                  |
| userCleanup            |
| userTracking           |
| users                  |
| virus                  |
+------------------------+
49 rows in set (0.00 sec)

george1421

Hmmm…

I think what I would do on this is look into the /opt/fog/.fogsettings file. That will have the parameters saved by the installer and the password selected for the mysql database. Then try to login to mysql using the FOG SQL password.

with mysql -u fog fog -p
That reads use user fog to connect to the fog database and prompt me for the password.

See if you can get logged into the mysql database. If you get logged in then do something like show tables; to list all of the fog related tables.

[edit] wait this doesn’t work on my fog server. I can’t test it right now I have to run [/edit]

sudburr

No proxy server. Again, 4/5 worked, but number 5 is not alive.

george1421

Is your FOG server behind a proxy server? I have seen where the fog server is behind a proxy server and you either use the environ variables or update wgetrc with proxy settings. This will cause the backup script to fail.

sudburr

So far I’ve tried updating to 7102, 7104 and 7106 on this server and it continues to fail at Backing up database.

My fresh iron build using the same source files and installed identically, works. yargen.

A little help? What has broken on this naughty server?

sudburr

I’m building a duplicate server right now from the same source files to see what shakes out.

build 7102 'Backing up database ... failed'

150
Online

10.4k
Users

16.4k
Topics

150.5k
Posts

build 7102 'Backing up database ... failed'

150Online

10.4kUsers

16.4kTopics

150.5kPosts

150
Online

10.4k
Users

16.4k
Topics

150.5k
Posts