Access denied for root@localhost

[Quazz]

So, I upgraded to Ubuntu 16.04 and as usual Mysql fails to configure itself during the upgrade.

So I removed all mysql with remove --purge and tried to let the fog installer take care of it, but unfortunately fog does not include the mysql-server installation in the script.

Therefore I tried to manually install mysql-server, which went fine without errors, but now fog gives the access denied for ‘root’@‘localhost’ error.

I then went into ssh terminal and tried to do mysql -u root, which also gives this message.

I was never given the option to enter a mysql root password (and have never had one).

I’ve already tried to remove the password using mysql commands, but it doesn’t change any rows (it does match one), which indicates there’s no password.

So not sure if something changed in ubuntu or the new mysql but it’s not working like before at any rate.

[eleder_gs]

Hi, I have the same problem when “install/upgrade” the database in a fresh install

[Tom Elliott]

I’m not aware of there being an issue at all. The mysql-server script is in the installer in lib/ubuntu/config.sh.

If, for whatever reason, it cannot find the mysql-server in the list, it will check for

sqlserverlist="mysql-server mariadb-server mariadb-galera-server MariaDB-server MariaDB-Galera-server"

If none of those exist, then something else has gone horribly wrong.

[Quazz]

@Tom-Elliott mysql-server is most definitely installed, it just isn’t able to login into mysql as a regular user, only using sudo or as root.

I assume FOG tries to login as a regular user (fog user most likely?) which fails like it does when I try it manually of course.

Would it be possible to simply install mariadbserver and try that?

[Sebastian Roth]

@Quazz said:

So I removed all mysql with remove --purge

AFAIK even purging the package does not remove /var/lib/mysql and therefore does not remove the mysql user tables with all your passwords set.

Try mysql -u root -p and it will ask you for a password!

[Quazz]

@Sebastian-Roth The problem is, there is no password set!

[Sebastian Roth]

@Quazz Well then it shouldn’t be a problem for the web gui to connect?! Can you please show us the exact error you see so we might have a better idea on where to dig.

[Quazz]

@Sebastian-Roth At home now, but the error I got on the fog schema updater was :

Access denied for ‘root’@‘localhost’. Please check if the database is running.

Using sudo mysql -u root through ssh gave me access no problem.

Using mysql -u root gave the Access denied for ‘root’@‘localhost’ error (I believe it was number 1696 or so?)

But your comment gave me an idea to move the var/lib/mysql and reinstall.

Tom had a brief join.me session before my internet cut out and after that I had to sort out our dhcp server first since apparmor didn’t allow it to create new dhcp lease files (still doesn’t, but it’s in complain mode now), so we’re guessing this error has something to do with ubuntu 16.04 specifically.

[Sebastian Roth]

@Quazz Confirmed (ubuntu) issue:

Checking if update is needed.
Checking server version.
Error: Server version (5.5.49-0ubuntu0.14.04.1) does not match with the version of
the server (5.7.12) with which this program was built/distributed. You can
use --skip-version-check to skip this check.
mysql_upgrade failed with exit status 3
dpkg: error processing package mysql-server-5.7 (--configure):
 subprocess installed post-installation script returned error exit status 1
...
dpkg: dependency problems prevent configuration of mysql-server:
 mysql-server depends on mysql-server-5.7; however:
  Package mysql-server-5.7 is not configured yet.

dpkg: error processing package mysql-server (--configure):
 dependency problems - leaving unconfigured
...
Errors were encountered while processing:
 mysql-server-5.7
 mysql-server
                                         
Current status: 0 updates [-272].

I am still not exactly sure what’s going on. Interestingly enough I seem to have solved this just by stopping and starting the mysql service (service mysql stop; service mysql start) as it did not complain about the version mismatch anymore when I tried running mysql_upgrade by hand after the service restart:

# su -s /bin/bash mysql
$ mysql_upgrade
Checking if update is needed.
This installation of MySQL is already upgraded to 5.7.12, use --force if you still need to run mysql_upgrade

But I seem to be caught in a “deadlock” now because I cannot clean up the install via aptitude -f install as it always fails when trying to talk to the older 5.5.49 mysql that is still running. The problem as I understand it is that the old DB needs to be up because the installer script runs mysql_upgrad which would fail if no DB is running. On the other hand if the old DB is up the new one cannot start because the files are locked. This is absolutely stupid. Why doesn’t it shutdown the old DB before trying to startup the new on?!?!

EDIT: I finally got around this by opening two shells. First run aptitude -f install and wait till you saw the message “This installation of MySQL is already upgraded to 5.7.12…”, then run service mysql stop to shutdown the old DB. Now I finally have only the new DB installed and starting/stopping using the init scripts is working fine again (new DB as I can see in /var/log/mysql/error.log)!

[Sebastian Roth]

@Quazz From what you said I guess you already got past the mysql startup and conversation issue. It’s just the login that does not work. Found this:

If you install 5.7 and don’t provide a password to the root user, it will use the auth_socket plugin. That plugin doesn’t care and doesn’t need a password. It just checks if the user is connecting using a UNIX socket and then compares the username.

So the simple fix is to login as root via command line client and run ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY ''; (will set an empty password again and more importantly sets it to native password login). Then go back to the web gui and I am pretty sure it won’t come up with the schemaupdater!

EDIT: I ran into “big trouble” with FOG and MySQL 5.7… You won’t be able to create any new DB entries (new hosts, groups, tasks) as the auto increment ID field seems to behave a little different.

[Quazz]

I think maybe Ubuntu pushed a patch to fix the upgrade thing? I can’t reproduce it any longer anyway.

Anyway, after running that command inside of mysql I can now connect as a regular user to the database and the schema updater no longer shows up!

[Quazz]

As you say, it seems unable to create tasks and such, which is troublesome.

Also, this apache error is generated every 0.25seconds or so:

[Wed Apr 27 10:43:47.795413 2016] [:error] [pid 20862] [client 192.168.1.31:55867] PHP Warning: implode(): Invalid arguments passed in /var/www/fog/lib/fog/eventmanager.class.php on line 67 (I’m guessing it’s complaining about empty string?)

Looking at the file it seems to be related to plugins (I have none installed currently)

Also, the FOG Settings page shows no settings anymore.

Guess I jumped the gun a bit with upgrading to 16.04 heh.

[Quazz]

FOG Services are dumping a ton of php errors, I’m guessing Ubuntu changed something about php/apache as well?

[Wed Apr 27 11:38:44.437447 2016] [evasive20:error] [pid 3521] [client 192.168.1.154:56732] client denied by server configuration: /var/www/fog/status/getfiles.php

Seems like there’s some security changes getting in the way?

[Quazz]

I think ubuntu renamed php packages for this release. I purged php and apache and now the foginstaller is unable to find packages with php5 in its name. Looks like it only supports php 7!

Looks like this is turning into the great Ubuntu 16.04 bug thread heh.

edit: following Tom’s instructions from the php 7 thread https://forums.fogproject.org/topic/6285/php-7-0-0-finally-released/16

Results in the installer being unable to add the repositories (probably because it still refers to the old repositories rather than the new ones)

Ubuntu now refers to php7 packages as simply php it seems! Which brings it in line with Redhat systems.

Also, Apache2 evasive mod has blacklisted my IP lol.

Installer now gets stuck on stopping the web service.

php-msqli was missing so had to manually install it. Installer still stuck though.

Found this in the installer error log:

Failed to stop php5-fpm.service: Unit php5-fpm.service not loaded.

Tries to stop a service that’s of course not installed causing it to fail.

Fixed it using Tom’s instructions of the php 7.0 thread but changing it slightly:

php_ver="7.0"
php_verAdds="-7.0"
repo="php"

The repo is simply php, but the packages can still be referred to as 7.0, however apt will generally pick the correct php package by simply referring to them as php.

The installer now makes it to Backing up database where it fails!

--2016-04-27 13:32:39--  http://192.168.1.154/fog//management/export.php?type=sql
Verbinding maken met 192.168.1.154:80... verbonden.
HTTP-verzoek is verzonden; wachten op antwoord... 500 Internal Server Error
2016-04-27 13:32:39 Fout 500: Internal Server Error.

Apache log:

[Wed Apr 27 13:47:21.468247 2016] [:error] [pid 20798] [client 192.168.1.31:50418] PHP Warning:  implode(): Invalid arguments passed in /var/www/fog/lib/fog/eventmanager.class.php on line 67
[Wed Apr 27 13:47:21.470895 2016] [:error] [pid 20798] [client 192.168.1.31:50418] PHP Fatal error:  Uncaught Error: Call to undefined function mb_convert_encoding() in /var/www/fog/lib/fog/fogbase.class.php:540\nStack trace:\n#0 /var/www/fog/lib/fog/fogcore.class.php(88): FOGBase::getSetting('FOG_VIEW_DEFAUL...')\n#1 /var/www/fog/commons/init.php(149): FOGCore::setSessionEnv()\n#2 /var/www/fog/commons/base.inc.php(10): require_once('/var/www/fog/co...')\n#3 /var/www/fog/service/servicemodule-active.php(2): require_once('/var/www/fog/co...')\n#4 {main}\n  thrown in /var/www/fog/lib/fog/fogbase.class.php on line 540
[Wed Apr 27 13:47:21.525371 2016] [:error] [pid 20794] [client 192.168.1.31:50419] PHP Fatal error:  Uncaught Error: Call to undefined function mb_convert_encoding() in /var/www/fog/commons/init.php:44\nStack trace:\n#0 [internal function]: Initiator::{closure}('node')\n#1 /var/www/fog/commons/init.php(46): array_map(Object(Closure), Array)\n#2 /var/www/fog/commons/init.php(137): Initiator::startInit()\n#3 /var/www/fog/commons/base.inc.php(10): require_once('/var/www/fog/co...')\n#4 /var/www/fog/management/index.php(2): require('/var/www/fog/co...')\n#5 {main}\n  thrown in /var/www/fog/commons/init.php on line 44

[Tom Elliott]

@Quazz Sounds, to me, that you may need to install php7.0-mbstring?

[Quazz]

@Tom-Elliott It was indeed not installed! Installation now completes!

Still getting the same issues as before on the webui however, which I’m guessing has a lot to do with php7 and less with anything else?

[Quazz]

@Tom-Elliott Tried to update to 7334.

Installer fails to update the database. No errors to be found in the installer error logs.

[Sebastian Roth]

@Quazz said:

Still getting the same issues

Which issues/errors exactly? The mysql 5.7 stuff or what?

[Hanz]

@Sebastian-Roth

 * Backing up database.........................................OK
 * Updating Database...........................................Failed!
[bcs@fog-server bin]$ 

Updating consistently fails
and web ui is not working.

The 10.72.3.50 page isn’t working

10.72.3.50 is currently unable to handle this request.
500

bcs@fog-server ~]$ sudo mysqlcheck -r fog
[sudo] password for bcs: 
mysqlcheck: Got error: 1040: Too many connections when trying to connect
[bcs@fog-server ~]$ 

max_connections set to 2500 in /etc/my.cnf
Possibly connections not closing correctly ?
Updates on nodes worked flawlessly BTW

Processor usage is pegged at 100 for all four cores as well as many many httpd processes

[Sebastian Roth]

@Hanz I cannot see how this is related to the topic of Ubuntu 16.04 mysql issues!?! Please help me.