Database fails to initialize 2

[syschuck]

@syschuck; Uhh-oh. This looks like a serious bug in the installer. I just installed the latest fog from github on a freshly installed Ubuntu 16.04 LTS and it too has the exact same issue with the database not getting build during the install.

[Wayne Workman]

@syschuck I’ve been sorta working on Ubuntu 16 here: https://forums.fogproject.org/topic/8338/fresh-clean-ubuntu-16-with-fog-trunk
More like feeding the Devs what I find lol.

[Quazz]

@syschuck You could try checking out the error logs in bin/error_logs (of your svn folder). There should be a log for each attempted version install.

[syschuck]

@Quazz. That is part of the problem. There are no errors that are reported. The install script runs perfectly up to the point where it say;

• You still need to install/update your database schema.

• This can be done by opening a web browser and going to:

  http://192.168.1.4/fog/management

• Press [Enter] key when database is updated/installed.

When you do go to http://192.168.1.4/fog/management your get the message:

Your database connection appears to be invalid. FOG is unable to communicate with the database. There are many reasons why this could be the case. Please check your credentials in /var/www/fog/lib/fog/config.class.php. Also confirm that the database is indeed running. If credentials are correct, and the Database service is running, check to ensure your filesystem has enough space.

When I examine, the fog database hasn’t even been created in mysql.
That’s SVN 5950 on two freshly installed systems of Mageia 5, and Ubuntu 16.04.

[syschuck]

@syschuck. I’m still baffled with this bug. Just to make sure it wasn’t the OS or a library causing some issue, I when back to fog_1.2.0, downloaded the tar, built and installed it on Mageia 5. It worked perfect! So I thought, maybe the new fog 1.3.0 RC8 would be different if there was already a database installed. It didn’t. I’m trying now to go-back to a SVN revision where the SQL initialization worked.

[Wayne Workman]

@syschuck Find the line that creates the DB, and run it manually. Echo out any variables the line is using so you can construct the command to run.

[syschuck]

@Wayne-Workman. Solved! I’ve found something. I had to turn on debugging by changing two lines in packages/web/lib/fog/fogbase.class.php.

protected static $debug = false;
protected static $info = false;

Changing those to true opens a large amount of debug info.

The first line I see is the following;
FOG DEBUG: PDODB: Failed to connect: SQLSTATE[HY000] [2002] Connection refused
Several lines later.
FOG DEBUG: User: isValid Failed: Error: Invalid ID

There are also a large number of
FOG DEBUG: PDODB: Failed to query: Cannot connect to database
FOG DEBUG: PDODB: Failed to fetch: No query result, use query() first
FOG DEBUG: PDODB: Failed to get: No connection to the database

That might make since this is a first time install and the database doesn’t exist. This is using user = root and password = ‘’. I tested mysql with $ mysql -u root (from a ordinary user account) and was successfully logged in as mysql root without issue.

The web eventually does report:

Database Schema Installer / Updater
No connection available

So I suspect it’s not ever connecting the mysql database. PDODB: Failed to connect: SQLSTATE[HY000] [2002] Connection refused . Well a little googling shows that this error occurs when using the wrong host in mysql. We’ve been using an IP adress 127.0.0.1 as the hostname instead of ‘localhost’. So add;

echo “snmysqlhost=‘localhost’” >> lib/common/config.sh

fixed it!

So in otherwords, mysql was using user ‘root’@‘127.0.0.1’ instead of user ‘root’@‘localhost’.

[Wayne Workman]

@syschuck That’s interesting. I think it’s possible to pass variables to the installer. I’ll try that with Ubuntu 16 later.

[Tom Elliott]

@syschuck I usually use 127.0.0.1 so persistent connections can work, as I saw somewhat major issues using persistent connections and socket connections. Glad it was able to be fixed, but why does it matter if you use localhost vs. 127.0.0.1? I know one is socket (localhost) and the other is tcp (127.0.0.1), but why isn’t the default allowing creation of the root user to BOTH addresses? Interesting find. I’ll change the installer code to use localhost rather than 127.0.0.1 if the snmysqlhost field is blank.

[Wayne Workman]

@Tom-Elliott But, did we not have a mess with Ubuntu on this very issue before, and you chose to use 127.0.0.1 because of it?

I’d ask to only set it as localhost for Mangeia 5.

I’ll test this setting also with Ubuntu 16 as it suffers from the DB not being created.

[Sebastian Roth]

@Tom-Elliott said:

but why isn’t the default allowing creation of the root user to BOTH addresses?

Mysql does not automatically allow access to the reverse lookup of a hostname or vice versa. Even full qualified vs. short hostname might cause trouble. This is just the way mysql does things. You might want to look into adding both localhost and 127.0.0.1 but then there is the mess of keeping the passwords in sync.

[Tom Elliott]

@Sebastian-Roth The issue, as I’m seeing it, is somewhere there was a change. Maybe this 127.0.0.1 vs. localhost change (I don’t care, at this point, for FQDN vs. short hostname) happened in a different version of mysql? I’ve never seen an issue using 127.0.0.1 or localhost on mysql, until now.

In either case, using localhost shouldn’t pose an issue, and the only reason, as I stated earlier, that I started using tcp vs. socket was solely for persistent connections. Mind you this was still when I was using mysqli as the method of managing connections to the DB and I now use PDO which might have some better handling for this type of thing.

[Sebastian Roth]

@Tom-Elliott What I mean is explained here: http://serverfault.com/questions/544187/cant-connect-to-mysql-using-localhost-but-using-127-0-0-1-its-ok

mysql> select user,host from mysql.user;
+------+--------------------------------+
| user | host                           |
+------+--------------------------------+
| root | 127.0.0.1                      | 
| root | ::1                            |
| root | localhost                      |
+------+--------------------------------+
3 rows in set (0.00 sec)

As you can see there are different entries for the same user but different host entries in the mysql user table.

[Tom Elliott]

@Sebastian-Roth I understand, just I’m used to them all being ready to use at the same time when the initial installation occurs.

[syschuck]

@Tom-Elliott I think I’ve found the explanation for this behavior. It’s controlled by an option in the mysql configuration /etc/my.cnf. The option is controlled by the variable skip-network which on the MariaDB on Mageia 5 and Ubuntu 16 is enabled. Apparently this is a security enhancement. If the skip-network line is remarked out in /etc/my.cnf Fog has no problems with 127.0.0.1 (or ::1 ip6 for that matter).

It raises the question of how FOG should handle this. As newer security conscious distros come out, I suspect this will come up more and more.

[Tom Elliott]

@syschuck I’ve updated the installer to default to localhost as a “safety” of course you can still set your own host after, or inline with the installer.

I believe the variable is:

snmysqlhost='127.0.0.1'

Of course you can run the installer as:

snmysqlhost='localhost' ./installfog.sh -y

[syschuck]

@Tom-Elliott Hi Tom, here is a code fragment that will check /etc/my.cnf for the skip-networking flag and set ${snmysqlhost} appropriately. It may not always get it but I think my.cnf is pretty standard for mysql and MariaDB.

#!/bin/sh

mysqlcfg=/etc/my.cnf
snmysqlhost='127.0.0.1' 
if [ -f ${mysqlcfg} ]; then
  W=`grep skip-networking ${mysqlcfg} | sed "/^\s*\;/d;s/\s*\;[^\"']*$//" | sed "/^\s*#/d;s/\s*#[^\"']*$//"`
  if [ ! -z $W ]; then
    snmysqlhost='localhost' 
  fi
fi
echo ${snmysqlhost}
#
./installfog.sh -y

Tom, I think you can mark this one as solved.

[Tom Elliott]

@syschuck Because I always default to localhost, I think this can also be solved, however your issue brings up another potential downfall in that your patch only changes to snmysqlhost to localhost if skip-networking is on. However, if skip-networking is on, Storage Nodes won’t be able to communicate to the DB because we’ve essentially disabled networking access.

I propose this:

for mysqlconf in $(grep -rl '.*skip-networking' /etc); do
    sed -i '/.*skip-networking/ s/^#*/#/' -i $mysqlconf >>$workingdir/error_logs/fog_error_${version}.log 2>&1
done

This method, I think, is better in that I only comment the skip networking lines in any found mysql conf files (as I’m not aware of other configs having the skip-networking as a config). It means I don’t guess which files to edit as the return will only give us matching files.

I already do something similar to this for bind-address (at least for ARCH and have moved them both to happen on any system.

Hopefully this makes sense.

[syschuck]

@Tom-Elliott That makes sense. I think there is also a /etc/my.cnf.d under ubuntu and mageia that skip-networking could be hidden under. So seems reasonable. Be aware, my.cnf has two comment characters, ‘#’ and ‘;’. That’s why my sed thing was so contorted.

[Tom Elliott]

@syschuck But I can guarantee the comment is done with a ‘#’ anyway.

Unless the files are not interchangable.