No Database Connection after 1.5.7 update [Debian]

[Dan_Ansel]

Hello,

I’m running into some database connectivity issue after updating FoG.
I can’t start the mysql service anymore.
When I check what happened with the journalctl -xe command and I get this :

sept. 19 14:54:43 [server] mysqld[9653]: 2019-09-19 14:54:43 140399207951744 [Note] /usr/sbin/mysqld (mysqld 10.1.38-MariaDB-0+deb9u1) starting as process 9653 …
sept. 19 14:54:43 [server] systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
sept. 19 14:54:43 [server] systemd[1]: Failed to start MariaDB 10.1.38 database server.
sept. 19 14:54:43 [server] systemd[1]: mariadb.service: Unit entered failed state.
sept. 19 14:54:43 [server] systemd[1]: mariadb.service: Failed with result ‘exit-code’.

I read on the forum about the database connection issue but it’s all about Ubuntu, and I’m running Debian 9.
I would really appreciate any help to get my database up and running again.

Thank You

Dan

Edit : I forgot the write down the error I get when trying to start MySQL :
Job for mariadb.service failed because the control process exited with error code.
See “systemctl status mariadb.service” and “journalctl -xe” for details.
The /var/run/mysqld directory is empty too

[Quazz]

Please give the output of tail -50 /var/log/mariadb/mariadb.log (not sure the path is same on Debian)

[Dan_Ansel]

I don’t have a /var/log/mariadb directory, only /var/log/mysql

Here’s what inside /var/log/mysql/error.log

2019-09-19 15:43:54 140433339829632 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.

2019-09-19 15:43:54 140433339829632 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2019-09-19 15:43:54 140433339829632 [Note] InnoDB: The InnoDB memory heap is disabled
2019-09-19 15:43:54 140433339829632 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-09-19 15:43:54 140433339829632 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-09-19 15:43:54 140433339829632 [Note] InnoDB: Compressed tables use zlib 1.2.8
2019-09-19 15:43:54 140433339829632 [Note] InnoDB: Using Linux native AIO
2019-09-19 15:43:54 140433339829632 [Note] InnoDB: Using SSE crc32 instructions
2019-09-19 15:43:54 140433339829632 [ERROR] mysqld: Can’t create/write to file ‘/tmp/ib4RWGdN’ (Errcode: 13 “Permission denied”)
2019-09-19 15:43:54 7fb92f55cd80 InnoDB: Error: unable to create temporary file; errno: 13
2019-09-19 15:43:54 140433339829632 [ERROR] Plugin ‘InnoDB’ init function returned error.
2019-09-19 15:43:54 140433339829632 [ERROR] Plugin ‘InnoDB’ registration as a STORAGE ENGINE failed.
2019-09-19 15:43:54 140433339829632 [Note] Plugin ‘FEEDBACK’ is disabled.
2019-09-19 15:43:54 140433339829632 [ERROR] Unknown/unsupported storage engine: InnoDB
2019-09-19 15:43:54 140433339829632 [ERROR] Aborting

2019-09-19 15:45:55 139812706057600 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.

2019-09-19 15:45:55 139812706057600 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2019-09-19 15:45:55 139812706057600 [Note] InnoDB: The InnoDB memory heap is disabled
2019-09-19 15:45:55 139812706057600 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-09-19 15:45:55 139812706057600 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-09-19 15:45:55 139812706057600 [Note] InnoDB: Compressed tables use zlib 1.2.8
2019-09-19 15:45:55 139812706057600 [Note] InnoDB: Using Linux native AIO
2019-09-19 15:45:55 139812706057600 [Note] InnoDB: Using SSE crc32 instructions
2019-09-19 15:45:55 139812706057600 [ERROR] mysqld: Can’t create/write to file ‘/tmp/ibDn1xvV’ (Errcode: 13 “Permission denied”)
2019-09-19 15:45:55 7f28aeadfd80 InnoDB: Error: unable to create temporary file; errno: 13
2019-09-19 15:45:55 139812706057600 [ERROR] Plugin ‘InnoDB’ init function returned error.
2019-09-19 15:45:55 139812706057600 [ERROR] Plugin ‘InnoDB’ registration as a STORAGE ENGINE failed.
2019-09-19 15:45:55 139812706057600 [ERROR] Unknown/unsupported storage engine: InnoDB
2019-09-19 15:45:55 139812706057600 [ERROR] Aborting

2019-09-19 15:50:17 140527244000640 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.

2019-09-19 15:50:17 140527244000640 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2019-09-19 15:50:17 140527244000640 [Note] InnoDB: The InnoDB memory heap is disabled
2019-09-19 15:50:17 140527244000640 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-09-19 15:50:17 140527244000640 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-09-19 15:50:17 140527244000640 [Note] InnoDB: Compressed tables use zlib 1.2.8
2019-09-19 15:50:17 140527244000640 [Note] InnoDB: Using Linux native AIO
2019-09-19 15:50:17 140527244000640 [Note] InnoDB: Using SSE crc32 instructions
2019-09-19 15:50:17 140527244000640 [ERROR] mysqld: Can’t create/write to file ‘/tmp/ib5rDzMm’ (Errcode: 13 “Permission denied”)
2019-09-19 15:50:17 7fcf0c75bd80 InnoDB: Error: unable to create temporary file; errno: 13
2019-09-19 15:50:17 140527244000640 [ERROR] Plugin ‘InnoDB’ init function returned error.
2019-09-19 15:50:17 140527244000640 [ERROR] Plugin ‘InnoDB’ registration as a STORAGE ENGINE failed.
2019-09-19 15:50:17 140527244000640 [Note] Plugin ‘FEEDBACK’ is disabled.
2019-09-19 15:50:17 140527244000640 [ERROR] Unknown/unsupported storage engine: InnoDB
2019-09-19 15:50:17 140527244000640 [ERROR] Aborting

For information, MySQL was installed during the fog installation.

[Quazz]

@Dan_Ansel Can you check permissions on /tmp ? Something like sudo ls -lah /tmp

It can’t create it’s temporary file required for startup, which might be the entire cause of the issue.

[Dan_Ansel]

There it is :

ls -lah /tmp
total 40M
drwxrwxr-x 12 root root 4,0K sept. 19 16:11 .
drwxr-xr-x 25 root root 4,0K sept. 19 14:20 …

I did not touch the permission on /tmp, is it safe to change them with chmod?

[Quazz]

@Dan_Ansel Yes, /tmp can be permissions 777 safely.

This error can also be caused by AppArmor I believe, but we’ll start here.

[Dan_Ansel]

It seems to have done it!
I’m running the updater again
It’s doing more than the first time, installing and updating packages.

I have absolutely no idea how the permission on the /tmp changed.

Thank You very much.

[Dan_Ansel]

It’s all good.
The database is up and running again.
Sorry for the inconvenience, I should’ve found out about the /tmp permissions shenanigans.

Thank You again.

Dan

[Quazz]

@Dan_Ansel No problem, there’s so much to think about, it can be hard to figure out what’s going wrong!