Access denied for root@localhost

Sebastian Roth

@Quazz Well then it shouldn’t be a problem for the web gui to connect?! Can you please show us the exact error you see so we might have a better idea on where to dig.

Quazz

@Sebastian-Roth At home now, but the error I got on the fog schema updater was :

Access denied for ‘root’@‘localhost’. Please check if the database is running.

Using sudo mysql -u root through ssh gave me access no problem.

Using mysql -u root gave the Access denied for ‘root’@‘localhost’ error (I believe it was number 1696 or so?)

But your comment gave me an idea to move the var/lib/mysql and reinstall.

Tom had a brief join.me session before my internet cut out and after that I had to sort out our dhcp server first since apparmor didn’t allow it to create new dhcp lease files (still doesn’t, but it’s in complain mode now), so we’re guessing this error has something to do with ubuntu 16.04 specifically.

Sebastian Roth

@Quazz Confirmed (ubuntu) issue:

Checking if update is needed.
Checking server version.
Error: Server version (5.5.49-0ubuntu0.14.04.1) does not match with the version of
the server (5.7.12) with which this program was built/distributed. You can
use --skip-version-check to skip this check.
mysql_upgrade failed with exit status 3
dpkg: error processing package mysql-server-5.7 (--configure):
 subprocess installed post-installation script returned error exit status 1
...
dpkg: dependency problems prevent configuration of mysql-server:
 mysql-server depends on mysql-server-5.7; however:
  Package mysql-server-5.7 is not configured yet.

dpkg: error processing package mysql-server (--configure):
 dependency problems - leaving unconfigured
...
Errors were encountered while processing:
 mysql-server-5.7
 mysql-server
                                         
Current status: 0 updates [-272].

I am still not exactly sure what’s going on. Interestingly enough I seem to have solved this just by stopping and starting the mysql service (service mysql stop; service mysql start) as it did not complain about the version mismatch anymore when I tried running mysql_upgrade by hand after the service restart:

# su -s /bin/bash mysql
$ mysql_upgrade
Checking if update is needed.
This installation of MySQL is already upgraded to 5.7.12, use --force if you still need to run mysql_upgrade

But I seem to be caught in a “deadlock” now because I cannot clean up the install via aptitude -f install as it always fails when trying to talk to the older 5.5.49 mysql that is still running. The problem as I understand it is that the old DB needs to be up because the installer script runs mysql_upgrad which would fail if no DB is running. On the other hand if the old DB is up the new one cannot start because the files are locked. This is absolutely stupid. Why doesn’t it shutdown the old DB before trying to startup the new on?!?!

EDIT: I finally got around this by opening two shells. First run aptitude -f install and wait till you saw the message “This installation of MySQL is already upgraded to 5.7.12…”, then run service mysql stop to shutdown the old DB. Now I finally have only the new DB installed and starting/stopping using the init scripts is working fine again (new DB as I can see in /var/log/mysql/error.log)!

Sebastian Roth

@Quazz From what you said I guess you already got past the mysql startup and conversation issue. It’s just the login that does not work. Found this:

If you install 5.7 and don’t provide a password to the root user, it will use the auth_socket plugin. That plugin doesn’t care and doesn’t need a password. It just checks if the user is connecting using a UNIX socket and then compares the username.

So the simple fix is to login as root via command line client and run ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY ''; (will set an empty password again and more importantly sets it to native password login). Then go back to the web gui and I am pretty sure it won’t come up with the schemaupdater!

EDIT: I ran into “big trouble” with FOG and MySQL 5.7… You won’t be able to create any new DB entries (new hosts, groups, tasks) as the auto increment ID field seems to behave a little different.

Quazz

I think maybe Ubuntu pushed a patch to fix the upgrade thing? I can’t reproduce it any longer anyway.

Anyway, after running that command inside of mysql I can now connect as a regular user to the database and the schema updater no longer shows up!

Quazz

As you say, it seems unable to create tasks and such, which is troublesome.

Also, this apache error is generated every 0.25seconds or so:

[Wed Apr 27 10:43:47.795413 2016] [:error] [pid 20862] [client 192.168.1.31:55867] PHP Warning: implode(): Invalid arguments passed in /var/www/fog/lib/fog/eventmanager.class.php on line 67 (I’m guessing it’s complaining about empty string?)

Looking at the file it seems to be related to plugins (I have none installed currently)

Also, the FOG Settings page shows no settings anymore.

Guess I jumped the gun a bit with upgrading to 16.04 heh.

Quazz

FOG Services are dumping a ton of php errors, I’m guessing Ubuntu changed something about php/apache as well?

[Wed Apr 27 11:38:44.437447 2016] [evasive20:error] [pid 3521] [client 192.168.1.154:56732] client denied by server configuration: /var/www/fog/status/getfiles.php

Seems like there’s some security changes getting in the way?

Quazz

I think ubuntu renamed php packages for this release. I purged php and apache and now the foginstaller is unable to find packages with php5 in its name. Looks like it only supports php 7!

Looks like this is turning into the great Ubuntu 16.04 bug thread heh.

edit: following Tom’s instructions from the php 7 thread https://forums.fogproject.org/topic/6285/php-7-0-0-finally-released/16

Results in the installer being unable to add the repositories (probably because it still refers to the old repositories rather than the new ones)

Ubuntu now refers to php7 packages as simply php it seems! Which brings it in line with Redhat systems.

Also, Apache2 evasive mod has blacklisted my IP lol.

Installer now gets stuck on stopping the web service.

php-msqli was missing so had to manually install it. Installer still stuck though.

Found this in the installer error log:

Failed to stop php5-fpm.service: Unit php5-fpm.service not loaded.

Tries to stop a service that’s of course not installed causing it to fail.

Fixed it using Tom’s instructions of the php 7.0 thread but changing it slightly:

php_ver="7.0"
php_verAdds="-7.0"
repo="php"

The repo is simply php, but the packages can still be referred to as 7.0, however apt will generally pick the correct php package by simply referring to them as php.

The installer now makes it to Backing up database where it fails!

--2016-04-27 13:32:39--  http://192.168.1.154/fog//management/export.php?type=sql
Verbinding maken met 192.168.1.154:80... verbonden.
HTTP-verzoek is verzonden; wachten op antwoord... 500 Internal Server Error
2016-04-27 13:32:39 Fout 500: Internal Server Error.

Apache log:

[Wed Apr 27 13:47:21.468247 2016] [:error] [pid 20798] [client 192.168.1.31:50418] PHP Warning:  implode(): Invalid arguments passed in /var/www/fog/lib/fog/eventmanager.class.php on line 67
[Wed Apr 27 13:47:21.470895 2016] [:error] [pid 20798] [client 192.168.1.31:50418] PHP Fatal error:  Uncaught Error: Call to undefined function mb_convert_encoding() in /var/www/fog/lib/fog/fogbase.class.php:540\nStack trace:\n#0 /var/www/fog/lib/fog/fogcore.class.php(88): FOGBase::getSetting('FOG_VIEW_DEFAUL...')\n#1 /var/www/fog/commons/init.php(149): FOGCore::setSessionEnv()\n#2 /var/www/fog/commons/base.inc.php(10): require_once('/var/www/fog/co...')\n#3 /var/www/fog/service/servicemodule-active.php(2): require_once('/var/www/fog/co...')\n#4 {main}\n  thrown in /var/www/fog/lib/fog/fogbase.class.php on line 540
[Wed Apr 27 13:47:21.525371 2016] [:error] [pid 20794] [client 192.168.1.31:50419] PHP Fatal error:  Uncaught Error: Call to undefined function mb_convert_encoding() in /var/www/fog/commons/init.php:44\nStack trace:\n#0 [internal function]: Initiator::{closure}('node')\n#1 /var/www/fog/commons/init.php(46): array_map(Object(Closure), Array)\n#2 /var/www/fog/commons/init.php(137): Initiator::startInit()\n#3 /var/www/fog/commons/base.inc.php(10): require_once('/var/www/fog/co...')\n#4 /var/www/fog/management/index.php(2): require('/var/www/fog/co...')\n#5 {main}\n  thrown in /var/www/fog/commons/init.php on line 44

Tom Elliott

@Quazz Sounds, to me, that you may need to install php7.0-mbstring?

Quazz

@Tom-Elliott It was indeed not installed! Installation now completes!

Still getting the same issues as before on the webui however, which I’m guessing has a lot to do with php7 and less with anything else?

Quazz

@Tom-Elliott Tried to update to 7334.

Installer fails to update the database. No errors to be found in the installer error logs.

Sebastian Roth

@Quazz said:

Still getting the same issues

Which issues/errors exactly? The mysql 5.7 stuff or what?

Hanz

@Sebastian-Roth

 * Backing up database.........................................OK
 * Updating Database...........................................Failed!
[bcs@fog-server bin]$

Updating consistently fails
and web ui is not working.

The 10.72.3.50 page isn’t working

10.72.3.50 is currently unable to handle this request.
500

bcs@fog-server ~]$ sudo mysqlcheck -r fog
[sudo] password for bcs: 
mysqlcheck: Got error: 1040: Too many connections when trying to connect
[bcs@fog-server ~]$

max_connections set to 2500 in /etc/my.cnf
Possibly connections not closing correctly ?
Updates on nodes worked flawlessly BTW

Processor usage is pegged at 100 for all four cores as well as many many httpd processes

Sebastian Roth

@Hanz I cannot see how this is related to the topic of Ubuntu 16.04 mysql issues!?! Please help me.

Quazz

@Sebastian-Roth Unable to create tasks, no fog settings being displayed, the php error spam.

Mysql seems to work fine now (assuming it’s not causing the other issues that is)

Quazz

Images are unable to deploy (input/output error).


[Wed Apr 27 17:02:54.713022 2016] [:error] [pid 9551] [client 192.168.1.73:55204] PHP Fatal error:  Uncaught Error: Call to a member function get() on null in /var/www/fog/lib/fog/bootmenu.class.php:380\nStack trace:\n#0 /var/www/fog/lib/fog/bootmenu.class.php(565): BootMenu->falseTasking('', Object(Image))\n#1 /var/www/fog/lib/fog/bootmenu.class.php(538): BootMenu->setTasking('12')\n#2 /var/www/fog/lib/fog/bootmenu.class.php(128): BootMenu->verifyCreds()\n#3 [internal function]: BootMenu->__construct(Object(Host))\n#4 /var/www/fog/lib/fog/fogbase.class.php(89): ReflectionClass->newInstance(Object(Host))\n#5 /var/www/fog/service/ipxe/boot.php(14): FOGBase::getClass('BootMenu', Object(Host))\n#6 {main}\n  thrown in /var/www/fog/lib/fog/bootmenu.class.php on line 380
[Wed Apr 27 17:05:12.363397 2016] [:error] [pid 6003] [client 192.168.1.73:43795] PHP Fatal error:  Uncaught Error: Call to a member function get() on null in /var/www/fog/lib/fog/bootmenu.class.php:380\nStack trace:\n#0 /var/www/fog/lib/fog/bootmenu.class.php(565): BootMenu->falseTasking('', Object(Image))\n#1 /var/www/fog/lib/fog/bootmenu.class.php(538): BootMenu->setTasking('12')\n#2 /var/www/fog/lib/fog/bootmenu.class.php(128): BootMenu->verifyCreds()\n#3 [internal function]: BootMenu->__construct(Object(Host))\n#4 /var/www/fog/lib/fog/fogbase.class.php(89): ReflectionClass->newInstance(Object(Host))\n#5 /var/www/fog/service/ipxe/boot.php(14): FOGBase::getClass('BootMenu', Object(Host))\n#6 {main}\n  thrown in /var/www/fog/lib/fog/bootmenu.class.php on line 380
[Wed Apr 27 17:06:28.521520 2016] [:error] [pid 8828] [client 192.168.1.73:6592] PHP Fatal error:  Uncaught Error: Call to a member function get() on null in /var/www/fog/lib/fog/bootmenu.class.php:380\nStack trace:\n#0 /var/www/fog/lib/fog/bootmenu.class.php(565): BootMenu->falseTasking('', Object(Image))\n#1 /var/www/fog/lib/fog/bootmenu.class.php(538): BootMenu->setTasking('12')\n#2 /var/www/fog/lib/fog/bootmenu.class.php(128): BootMenu->verifyCreds()\n#3 [internal function]: BootMenu->__construct(Object(Host))\n#4 /var/www/fog/lib/fog/fogbase.class.php(89): ReflectionClass->newInstance(Object(Host))\n#5 /var/www/fog/service/ipxe/boot.php(14): FOGBase::getClass('BootMenu', Object(Host))\n#6 {main}\n  thrown in /var/www/fog/lib/fog/bootmenu.class.php on line 380

The other php errors have no longer showed up since Tom’s tomfoolery

ITCC

you should dump Ubuntu for this:
https://www.turnkeylinux.org/lampstack

install is SO easy and the features of the stack are good too.

Quazz

Added subnet to evasive mod whitelist to get rid off a bunch of stupid errors (it kept on blocking getfiles.php)

Perhaps worth noting that both php7.0-mbstring and php-mbstring exist and are in fact different packages. (why…)

Sebastian Roth

I think we need to face 16.04 at some point to I am trying to get a wiki article together: https://wiki.fogproject.org/wiki/index.php?title=Ubuntu_16.04

NSmith

I found the solution for me in this article: https://www.percona.com/blog/2016/03/16/change-user-password-in-mysql-5-7-with-plugin-auth_socket/

basically comes down to the following as the fix :

sudo mysql -u root
ALTER USER ‘root’@‘localhost’ IDENTIFIED WITH mysql_native_password BY ‘password’;
exit

Access denied for root@localhost

238

11.6k

17.1k

154.6k