Upgraded ubuntu from 13.04 to 14.04 now Web console will not come up

Wayne Workman

@darkxeno that’s an issue… it means that the latest schema updater code can’t update something in earlier revisions to the current revision…

Look at your MySQL logs and Apache logs to try and see what it is… What time did the schema updater fail? Your original post was 3 hours ago.

darkxeno

@darkxeno said:

Wed Jul 08 08:30:04.533150 2015] [:error] [pid 22878] [client 10.0.10.146:51962] PHP Warning: mysqli::poll(): No stream arrays were passed in /var/www/fog/lib/db/MySQL.class.php on line 67, referer: http://10.0.0.22/fog/management/?node=schemaupdater

this is whats in the apache logs repeated a bunch of times

150708 8:29:15 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
150708 8:29:15 [Note] Plugin ‘FEDERATED’ is disabled.
150708 8:29:15 InnoDB: The InnoDB memory heap is disabled
150708 8:29:15 InnoDB: Mutexes and rw_locks use GCC atomic builtins
150708 8:29:15 InnoDB: Compressed tables use zlib 1.2.8
150708 8:29:15 InnoDB: Using Linux native AIO
150708 8:29:15 InnoDB: Initializing buffer pool, size = 128.0M
150708 8:29:15 InnoDB: Completed initialization of buffer pool
150708 8:29:15 InnoDB: highest supported file format is Barracuda.
150708 8:29:15 InnoDB: Waiting for the background threads to start
150708 8:29:16 InnoDB: 5.5.43 started; log sequence number 2579942
150708 8:29:16 [Note] Server hostname (bind-address): ‘127.0.0.1’; port: 3306
150708 8:29:16 [Note] - ‘127.0.0.1’ resolves to ‘127.0.0.1’;
150708 8:29:16 [Note] Server socket created on IP: ‘127.0.0.1’.
150708 8:29:16 [Warning] Found invalid password for user: ‘timesheet@localhost’; Ignoring user
150708 8:29:16 [Warning] Found invalid password for user: ‘sa@localhost’; Ignoring user
150708 8:29:16 [Warning] Found invalid password for user: ‘timesheet1@localhost’; Ignoring user
150708 8:29:16 [Note] Event Scheduler: Loaded 0 events
150708 8:29:16 [Note] /usr/sbin/mysqld: ready for connections.
Version: ‘5.5.43-0ubuntu0.14.04.1’ socket: ‘/var/run/mysqld/mysqld.sock’ port: 3306 (Ubuntu)
150708 8:37:22 [Note] /usr/sbin/mysqld: Normal shutdown

150708 8:37:22 [Note] Event Scheduler: Purging the queue. 0 events
150708 8:37:24 [Warning] /usr/sbin/mysqld: Forcing close of thread 1554 user: ‘root’

150708 8:37:24 [Warning] /usr/sbin/mysqld: Forcing close of thread 1465 user: ‘root’

150708 8:37:24 InnoDB: Starting shutdown…
150708 8:37:25 InnoDB: Shutdown completed; log sequence number 2593161
150708 8:37:25 [Note] /usr/sbin/mysqld: Shutdown complete

150708 8:37:25 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
150708 8:37:25 [Note] Plugin ‘FEDERATED’ is disabled.
150708 8:37:25 InnoDB: The InnoDB memory heap is disabled
150708 8:37:25 InnoDB: Mutexes and rw_locks use GCC atomic builtins
150708 8:37:25 InnoDB: Compressed tables use zlib 1.2.8
150708 8:37:25 InnoDB: Using Linux native AIO
150708 8:37:25 InnoDB: Initializing buffer pool, size = 128.0M
150708 8:37:25 InnoDB: Completed initialization of buffer pool
150708 8:37:25 InnoDB: highest supported file format is Barracuda.
150708 8:37:25 InnoDB: Waiting for the background threads to start
150708 8:37:26 InnoDB: 5.5.43 started; log sequence number 2593161
150708 8:37:26 [Note] Server hostname (bind-address): ‘127.0.0.1’; port: 3306
150708 8:37:26 [Note] - ‘127.0.0.1’ resolves to ‘127.0.0.1’;
150708 8:37:26 [Note] Server socket created on IP: ‘127.0.0.1’.
150708 8:37:26 [Warning] Found invalid password for user: ‘timesheet@localhost’; Ignoring user
150708 8:37:26 [Warning] Found invalid password for user: ‘sa@localhost’; Ignoring user
150708 8:37:26 [Warning] Found invalid password for user: ‘timesheet1@localhost’; Ignoring user
150708 8:37:26 [Note] Event Scheduler: Loaded 0 events
150708 8:37:26 [Note] /usr/sbin/mysqld: ready for connections.
Version: ‘5.5.43-0ubuntu0.14.04.1’ socket: ‘/var/run/mysqld/mysqld.sock’ port: 3306 (Ubuntu)
150708 8:48:45 [Note] /usr/sbin/mysqld: Normal shutdown

150708 8:48:45 [Note] Event Scheduler: Purging the queue. 0 events
150708 8:48:45 InnoDB: Starting shutdown…
150708 8:48:46 InnoDB: Shutdown completed; log sequence number 2593161
150708 8:48:46 [Note] /usr/sbin/mysqld: Shutdown complete

this is my mysql logs

syschuck

Just to let the @Developers know. It’s not an ubuntu thing. I’m having the exact same issue with Mageia 5.
(Mysql 10.0.19-MariaDB).

[Wed Jul 08 11:57:29.718411 2015] [:error] [pid 18958] [client 192.168.1.1:41436] PHP Warning: mysqli::mysqli(): (HY000/2002): Connection refused in /var/www/html/fog/lib/db/MySQL.class.php on line 37
[Wed Jul 08 11:58:18.468757 2015] [:error] [pid 22579] [client 192.168.1.1:41901] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 59

I’m going to check that my mysql password is cleared and try again. I also noticed that it didn’t create the ‘fog’ data base.

syschuck

I removed the fog database to start form scratch and I can confirm my earlier report. The fog database is not created. My apache log files show the following;

[Wed Jul 08 12:28:57.084063 2015] [mpm_prefork:notice] [pid 5036] AH00170: caught SIGWINCH, shutting down gracefully
[Wed Jul 08 12:28:58.587272 2015] [auth_digest:notice] [pid 8184] AH01757: generating secret for digest authentication …
[Wed Jul 08 12:28:59.147391 2015] [mpm_prefork:notice] [pid 8184] AH00163: Apache/2.4.10 (Unix) OpenSSL/1.0.2c PHP/5.6.10 mod_perl/2.0.8-dev Perl/v5.20.1 configured – resuming normal operations
[Wed Jul 08 12:28:59.147496 2015] [core:notice] [pid 8184] AH00094: Command line: ‘/usr/sbin/httpd -D FOREGROUND’
[Wed Jul 08 12:29:12.183867 2015] [:error] [pid 8190] [client 192.168.1.1:59748] PHP Warning: mysqli::mysqli(): (HY000/2002): Connection refused in /var/www/html/fog/lib/db/MySQL.class.php on line 37
[Wed Jul 08 12:29:12.184019 2015] [:error] [pid 8190] [client 192.168.1.1:59748] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 59
[Wed Jul 08 12:29:12.196499 2015] [:error] [pid 8190] [client 192.168.1.1:59748] PHP Warning: mysqli::mysqli(): (HY000/2002): Connection refused in /var/www/html/fog/lib/db/MySQL.class.php on line 37
[Wed Jul 08 12:29:12.196879 2015] [:error] [pid 8190] [client 192.168.1.1:59748] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 59
[Wed Jul 08 12:29:12.197176 2015] [:error] [pid 8190] [client 192.168.1.1:59748] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 59

Last line repeated 20+ times.

Wayne Workman

@syschuck Sounds like either MySQL isn’t running, or a password is set.

But, I assume these aren’t the case… $:-\$

syschuck

I triple checked mysql and root does not have a password and the whole sql is wide open to use.

[root@fog bin]# mysqladmin status
Uptime: 662 Threads: 1 Questions: 2 Slow queries: 0 Opens: 0 Flush tables: 1 Open tables: 63 Queries per second avg: 0.003

mysqladmin version
mysqladmin Ver 9.1 Distrib 10.0.19-MariaDB, for Linux on x86_64
Copyright 2000, 2015, Oracle, MariaDB Corporation Ab and others.

Server version 10.0.19-MariaDB
Protocol version 10
Connection Localhost via UNIX socket
UNIX socket /var/lib/mysql/mysql.sock
Uptime: 12 min 24 sec

This is a side note, I discovered while looking at the mysql issue.
Wow. Just did a ‘ps -ef | grep fog’ and found I had like 20+ instances of

root 7469 1 0 Jul07 ? 00:01:22 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager
root 7532 1 0 Jul07 ? 00:00:01 /usr/bin/php -q /opt/fog/service/FOGImageReplicator/FOGImageReplicator
root 7596 1 0 Jul07 ? 00:00:01 /usr/bin/php -q /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler
root 7662 1 0 Jul07 ? 00:00:01 /usr/bin/php -q /opt/fog/service/FOGSnapinReplicator/FOGSnapinReplicator

from all of my install attempts. That must be something minor. Installer shows;

Stopping FOGMulticastManager Service…/lib/common/functions.sh: line 444: 30965 Terminated $initdpath/$serviceItem stop > /dev/null 2>&1
OK
for each of the services.
For the Mageia5 variation on Redhat, this latest release removed the old systemV init compatibility and is now completely using systemd. So a changes needs to be made to bin/installfog.sh. It’s located down at the line; with
OSVersion=echo $OSVersion | cut -d '.' -f1

if [[ "$OSVersion" -ge 3 && "$linuxReleaseName" == +(*[Mm]'ageia'*) ]]; then
    systemctl="yes";
fi

So once systemctl is set, then it revealed another systemd issue in lib/common/functions.sh in that Mageia5 doesn’t use mariadb.service or mysql.service. It uses mysqld.service! So I made this change to the configureMySql() function.

    if [ "$systemctl" == "yes" ]; then
        if [[ "$linuxReleaseName" == +(*[Mm]'ageia'*) ]]; then
                systemctl="yes";
                systemctl enable mysqld.service >/dev/null 2>&1 && \
                systemctl restart mysqld.service >/dev/null 2>&1 && \
                systemctl status mysqld.service >/dev/null 2>&1
        else
                systemctl="yes";
                systemctl enable mariadb.service >/dev/null 2>&1 && \
                systemctl restart mariadb.service >/dev/null 2>&1 && \
                systemctl status mariadb.service >/dev/null 2>&1
                if [ "$?" != "0" ]; then
                        systemctl enable mysql.service >/dev/null 2>&1 && \
                        systemctl restart mysql.service >/dev/null 2>&1 && \
                        systemctl status mysql.service >/dev/null 2>&1
                fi
        fi

Well that has everything working with Mageia5 now. I’ll post the patches later. They are all minor things like above. However, none of that was the problem. It still redirects to http://192.168.1.1/fog/management/index.php?node=schemaupdater

with a blank white page.

Tom Elliott

@syschuck Can I help?

Wayne Workman

@syschuck said:

This is a side note, I discovered while looking at the mysql issue.
Wow. Just did a ‘ps -ef | grep fog’ and found I had like 20+ instances of

root 7469 1 0 Jul07 ? 00:01:22 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager
root 7532 1 0 Jul07 ? 00:00:01 /usr/bin/php -q /opt/fog/service/FOGImageReplicator/FOGImageReplicator
root 7596 1 0 Jul07 ? 00:00:01 /usr/bin/php -q /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler
root 7662 1 0 Jul07 ? 00:00:01 /usr/bin/php -q /opt/fog/service/FOGSnapinReplicator/FOGSnapinReplicator

from all of my install attempts. That must be something minor. Installer shows;

that’s how daemons (and daemon like services) are supposed to work. I’ve been learning about this recently. When a daemon starts, the first thing it does is fork itself (replicate itself into a background process in RAM), then chek if the fork was successful. If successful, the first copy shuts down while the second copy runs until either shutdown or a user stops it.

Each time you ran the installer, you created new instances of these services, and these services are a lot like actual daemons.

You can fix this by running “top” and then just issuing the “kill” command from there with the appropriate PID.

Or you can just reboot.

syschuck

@Tom-Elliott Hi Tom. How ya doing? Well, the guys at Mageia released a new distribution and so I thought I would test fog on it. It looks like this release has a few changes that need to be brought in to the fog installer. They are all minor name changes, and version conditionals. Would you like be to just post the patches on the forum?

The other problem that has me stumped, is that mysql is somehow not being updated or linked into. Everything looks good in the install;

Setting up and starting MySQL…OK
Backing up user reports…OK
- Did you leave the mysql password blank during install? (Y/n)
Setting up and starting Apache Web Server…OK

etc…

You still need to install/update your database schema.
This can be done by opening a web browser and going to:

  http://192.168.1.1/fog/management

Press [Enter] key when database is updated/installed.

And that redirects to http://192.168.1.1/fog/management/index.php?node=schemaupdater

In the apache logs, it show;

PHP Warning: mysqli::mysqli(): (HY000/2002): Connection refused in /var/www/html/fog/lib/db/MySQL.class.php on line 37

Which is;

if (!$this->link) $this->link = new mysqli(DATABASE_HOST, DATABASE_USERNAME, DATABASE_PASSWORD);

So I added a debug line;
printf(‘My info | %s | %s | %s <br>’, DATABASE_HOST, DATABASE_USERNAME, DATABASE_PASSWORD);

and it shows;

My info | p:127.0.0.1 | root |

So is the database host supposed to be ‘p:127.0.0.1’ ?

Tom Elliott

@syschuck Yes, it should be p:127.0.0.1, this puts the connections into “Persistent” mode for mysqli.

Wayne Workman

@syschuck said:

So is the database host supposed to be ‘p:127.0.0.1’ ?

There’s been a ton of issues lately with using the local loopback address with the database.

But, Mageia is Red Hat based and I’m unaware of any Red Hat distros except Mageia that suffer from this issue. It’s normally specific to Ubuntu 14.04 and newer…

Wayne Workman

@darkxeno

You know what I think?

I bet that there is some stupid setting somewhere in the mysql conf file that was grandfather’d in from 13.04 when you upgraded.

Can you give us the output of this:

cat /etc/mysql/my.cnf

Also, please ensure there is no database password… Log into MySQL via the CLI and see if it asks for a password. If it does ask for a password, give it the fog user’s password. if no luck, try with sudo.

su fog
mysql
use fog
select * from hosts;
exit

syschuck

@syschuck said:

printf(‘My info | %s | %s | %s <br>’, DATABASE_HOST, DATABASE_USERNAME, DATABASE_PASSWORD);

It could be: I did a little more slewthing, and the error is
‘Failed to connect: Issue working with the current DB, maybe it has not been created yet’

So, it looks like the issue is early when creating the fog db. Phpmyadmin shows that the fog db isn’t being created. It wouldn’t surprise me that ubuntu 14.04 and Mageia 5 are using the newer versions of mysql.
The Mageia 5 was released just a few weeks ago.

[fog@fog mysqld]$ id; mysql
uid=1000(fog) gid=1000(fog) groups=1000(fog)
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 1112
Server version: 10.0.19-MariaDB Mageia MariaDB Server

Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> use fog
ERROR 1049 (42000): Unknown database 'fog'
MariaDB [(none)]>

Wayne Workman

@syschuck What happens if you specified connecting to mysql using 127.0.0.1 ?

syschuck

@Wayne-Workman; Hi Wayne. Well I’ve looked all over and I’m at a complete loss for ideas on what may be causing the issue. It’s almost as if it’s skipping over the SQL when it creates the database and when it opens the database. I turned on php debugging, but nothing really shows up. Unfortunately work is piling up on me, so I may need to look at this another day.

I’m starting to get a lot of windows 8.1 machines that need the newer fog so I’m going to try some older OSs and see if that helps.

Wayne Workman

@syschuck

CentOS 7 or Fedora 22 Server … “The waters are fine, come on in.”

syschuck

@Wayne-Workman; Hi Wayne. Just installed Kabuntu 14.04 LTS and it worked perfectly. So for my immediate needs, I think I’ll be good to go. On the Mageia side, apparently this issue started in the previous release 4. I looked at the mysql 127.0.0.1 vs localhost issue and sure enough there was a host resolve problem. So #mysql -u root -h 127.0.0.1 -P 3306 would refuse, but #mysql -u root -h localhost -P 3306 worked. So I added the line: ‘bind-address = 0.0.0.0’ to /etc/my.cnf below the ‘skip-networking’ line.
That seemed to let everything in sql work without passwords. That got rid if all of the mysqli error message in the apache error_log. However, It’s still not creating the ‘fog’ database and I have the same blank webpage.
It doesn’t appear to be throwing any exceptions even though the database is never created. So I’m still at a loss. What user is being used for mysql? Is it the ‘fog’ user?

Upgraded ubuntu from 13.04 to 14.04 now Web console will not come up

262

11.6k

17.1k

154.6k