Access denied for root@localhost

[Quazz]

So, I upgraded to Ubuntu 16.04 and as usual Mysql fails to configure itself during the upgrade.

So I removed all mysql with remove --purge and tried to let the fog installer take care of it, but unfortunately fog does not include the mysql-server installation in the script.

Therefore I tried to manually install mysql-server, which went fine without errors, but now fog gives the access denied for ‘root’@‘localhost’ error.

I then went into ssh terminal and tried to do mysql -u root, which also gives this message.

I was never given the option to enter a mysql root password (and have never had one).

I’ve already tried to remove the password using mysql commands, but it doesn’t change any rows (it does match one), which indicates there’s no password.

So not sure if something changed in ubuntu or the new mysql but it’s not working like before at any rate.

[Sebastian Roth]

@Wayne-Workman Yes, force people to set a MySQL root password when installing. But it would still fail if people have MySQL server pre-installed (with empty password)… Other than that I don’t think we have any chance to hook into the auth_socket thing. See if you can find anything related to php and auth_socket

[Wayne Workman]

@Sebastian-Roth Is there any way to address this in the installer so that it works out of the box for Ubuntu 16?

[Sebastian Roth]

@Wayne-Workman Sorry, but that’s a bit of a different story as Ubuntu 16.04 comes with MySQL 5.7 which defaults to something called auth_socket in contrast to normal tcp localhost socket connection.

Issue and solution are known and documented here and here.

[Wayne Workman]

@NSmith that’s in our wiki. https://wiki.fogproject.org/wiki/index.php?title=Troubleshoot_MySQL#MySQL_credentials_for_FOG_Main_and_Storage_Nodes

[NSmith]

I found the solution for me in this article: https://www.percona.com/blog/2016/03/16/change-user-password-in-mysql-5-7-with-plugin-auth_socket/

basically comes down to the following as the fix :

sudo mysql -u root
ALTER USER ‘root’@‘localhost’ IDENTIFIED WITH mysql_native_password BY ‘password’;
exit

[Sebastian Roth]

I think we need to face 16.04 at some point to I am trying to get a wiki article together: https://wiki.fogproject.org/wiki/index.php?title=Ubuntu_16.04

[Quazz]

Added subnet to evasive mod whitelist to get rid off a bunch of stupid errors (it kept on blocking getfiles.php)

Perhaps worth noting that both php7.0-mbstring and php-mbstring exist and are in fact different packages. (why…)

[ITCC]

you should dump Ubuntu for this:
https://www.turnkeylinux.org/lampstack

install is SO easy and the features of the stack are good too.

[Quazz]

Images are unable to deploy (input/output error).

[Wed Apr 27 17:02:54.713022 2016] [:error] [pid 9551] [client 192.168.1.73:55204] PHP Fatal error:  Uncaught Error: Call to a member function get() on null in /var/www/fog/lib/fog/bootmenu.class.php:380\nStack trace:\n#0 /var/www/fog/lib/fog/bootmenu.class.php(565): BootMenu->falseTasking('', Object(Image))\n#1 /var/www/fog/lib/fog/bootmenu.class.php(538): BootMenu->setTasking('12')\n#2 /var/www/fog/lib/fog/bootmenu.class.php(128): BootMenu->verifyCreds()\n#3 [internal function]: BootMenu->__construct(Object(Host))\n#4 /var/www/fog/lib/fog/fogbase.class.php(89): ReflectionClass->newInstance(Object(Host))\n#5 /var/www/fog/service/ipxe/boot.php(14): FOGBase::getClass('BootMenu', Object(Host))\n#6 {main}\n  thrown in /var/www/fog/lib/fog/bootmenu.class.php on line 380
[Wed Apr 27 17:05:12.363397 2016] [:error] [pid 6003] [client 192.168.1.73:43795] PHP Fatal error:  Uncaught Error: Call to a member function get() on null in /var/www/fog/lib/fog/bootmenu.class.php:380\nStack trace:\n#0 /var/www/fog/lib/fog/bootmenu.class.php(565): BootMenu->falseTasking('', Object(Image))\n#1 /var/www/fog/lib/fog/bootmenu.class.php(538): BootMenu->setTasking('12')\n#2 /var/www/fog/lib/fog/bootmenu.class.php(128): BootMenu->verifyCreds()\n#3 [internal function]: BootMenu->__construct(Object(Host))\n#4 /var/www/fog/lib/fog/fogbase.class.php(89): ReflectionClass->newInstance(Object(Host))\n#5 /var/www/fog/service/ipxe/boot.php(14): FOGBase::getClass('BootMenu', Object(Host))\n#6 {main}\n  thrown in /var/www/fog/lib/fog/bootmenu.class.php on line 380
[Wed Apr 27 17:06:28.521520 2016] [:error] [pid 8828] [client 192.168.1.73:6592] PHP Fatal error:  Uncaught Error: Call to a member function get() on null in /var/www/fog/lib/fog/bootmenu.class.php:380\nStack trace:\n#0 /var/www/fog/lib/fog/bootmenu.class.php(565): BootMenu->falseTasking('', Object(Image))\n#1 /var/www/fog/lib/fog/bootmenu.class.php(538): BootMenu->setTasking('12')\n#2 /var/www/fog/lib/fog/bootmenu.class.php(128): BootMenu->verifyCreds()\n#3 [internal function]: BootMenu->__construct(Object(Host))\n#4 /var/www/fog/lib/fog/fogbase.class.php(89): ReflectionClass->newInstance(Object(Host))\n#5 /var/www/fog/service/ipxe/boot.php(14): FOGBase::getClass('BootMenu', Object(Host))\n#6 {main}\n  thrown in /var/www/fog/lib/fog/bootmenu.class.php on line 380

The other php errors have no longer showed up since Tom’s tomfoolery

[Quazz]

@Sebastian-Roth Unable to create tasks, no fog settings being displayed, the php error spam.

Mysql seems to work fine now (assuming it’s not causing the other issues that is)

[Sebastian Roth]

@Hanz I cannot see how this is related to the topic of Ubuntu 16.04 mysql issues!?! Please help me.

[Hanz]

@Sebastian-Roth

 * Backing up database.........................................OK
 * Updating Database...........................................Failed!
[bcs@fog-server bin]$ 

Updating consistently fails
and web ui is not working.

The 10.72.3.50 page isn’t working

10.72.3.50 is currently unable to handle this request.
500

bcs@fog-server ~]$ sudo mysqlcheck -r fog
[sudo] password for bcs: 
mysqlcheck: Got error: 1040: Too many connections when trying to connect
[bcs@fog-server ~]$ 

max_connections set to 2500 in /etc/my.cnf
Possibly connections not closing correctly ?
Updates on nodes worked flawlessly BTW

Processor usage is pegged at 100 for all four cores as well as many many httpd processes

[Sebastian Roth]

@Quazz said:

Still getting the same issues

Which issues/errors exactly? The mysql 5.7 stuff or what?

[Quazz]

@Tom-Elliott Tried to update to 7334.

Installer fails to update the database. No errors to be found in the installer error logs.

[Quazz]

@Tom-Elliott It was indeed not installed! Installation now completes!

Still getting the same issues as before on the webui however, which I’m guessing has a lot to do with php7 and less with anything else?

[Tom Elliott]

@Quazz Sounds, to me, that you may need to install php7.0-mbstring?

[Quazz]

I think ubuntu renamed php packages for this release. I purged php and apache and now the foginstaller is unable to find packages with php5 in its name. Looks like it only supports php 7!

Looks like this is turning into the great Ubuntu 16.04 bug thread heh.

edit: following Tom’s instructions from the php 7 thread https://forums.fogproject.org/topic/6285/php-7-0-0-finally-released/16

Results in the installer being unable to add the repositories (probably because it still refers to the old repositories rather than the new ones)

Ubuntu now refers to php7 packages as simply php it seems! Which brings it in line with Redhat systems.

Also, Apache2 evasive mod has blacklisted my IP lol.

Installer now gets stuck on stopping the web service.

php-msqli was missing so had to manually install it. Installer still stuck though.

Found this in the installer error log:

Failed to stop php5-fpm.service: Unit php5-fpm.service not loaded.

Tries to stop a service that’s of course not installed causing it to fail.

Fixed it using Tom’s instructions of the php 7.0 thread but changing it slightly:

php_ver="7.0"
php_verAdds="-7.0"
repo="php"

The repo is simply php, but the packages can still be referred to as 7.0, however apt will generally pick the correct php package by simply referring to them as php.

The installer now makes it to Backing up database where it fails!

--2016-04-27 13:32:39--  http://192.168.1.154/fog//management/export.php?type=sql
Verbinding maken met 192.168.1.154:80... verbonden.
HTTP-verzoek is verzonden; wachten op antwoord... 500 Internal Server Error
2016-04-27 13:32:39 Fout 500: Internal Server Error.

Apache log:

[Wed Apr 27 13:47:21.468247 2016] [:error] [pid 20798] [client 192.168.1.31:50418] PHP Warning:  implode(): Invalid arguments passed in /var/www/fog/lib/fog/eventmanager.class.php on line 67
[Wed Apr 27 13:47:21.470895 2016] [:error] [pid 20798] [client 192.168.1.31:50418] PHP Fatal error:  Uncaught Error: Call to undefined function mb_convert_encoding() in /var/www/fog/lib/fog/fogbase.class.php:540\nStack trace:\n#0 /var/www/fog/lib/fog/fogcore.class.php(88): FOGBase::getSetting('FOG_VIEW_DEFAUL...')\n#1 /var/www/fog/commons/init.php(149): FOGCore::setSessionEnv()\n#2 /var/www/fog/commons/base.inc.php(10): require_once('/var/www/fog/co...')\n#3 /var/www/fog/service/servicemodule-active.php(2): require_once('/var/www/fog/co...')\n#4 {main}\n  thrown in /var/www/fog/lib/fog/fogbase.class.php on line 540
[Wed Apr 27 13:47:21.525371 2016] [:error] [pid 20794] [client 192.168.1.31:50419] PHP Fatal error:  Uncaught Error: Call to undefined function mb_convert_encoding() in /var/www/fog/commons/init.php:44\nStack trace:\n#0 [internal function]: Initiator::{closure}('node')\n#1 /var/www/fog/commons/init.php(46): array_map(Object(Closure), Array)\n#2 /var/www/fog/commons/init.php(137): Initiator::startInit()\n#3 /var/www/fog/commons/base.inc.php(10): require_once('/var/www/fog/co...')\n#4 /var/www/fog/management/index.php(2): require('/var/www/fog/co...')\n#5 {main}\n  thrown in /var/www/fog/commons/init.php on line 44

[Quazz]

FOG Services are dumping a ton of php errors, I’m guessing Ubuntu changed something about php/apache as well?

[Wed Apr 27 11:38:44.437447 2016] [evasive20:error] [pid 3521] [client 192.168.1.154:56732] client denied by server configuration: /var/www/fog/status/getfiles.php

Seems like there’s some security changes getting in the way?

[Quazz]

As you say, it seems unable to create tasks and such, which is troublesome.

Also, this apache error is generated every 0.25seconds or so:

[Wed Apr 27 10:43:47.795413 2016] [:error] [pid 20862] [client 192.168.1.31:55867] PHP Warning: implode(): Invalid arguments passed in /var/www/fog/lib/fog/eventmanager.class.php on line 67 (I’m guessing it’s complaining about empty string?)

Looking at the file it seems to be related to plugins (I have none installed currently)

Also, the FOG Settings page shows no settings anymore.

Guess I jumped the gun a bit with upgrading to 16.04 heh.