Unmodifiable DHCP-Server + Fog / Updating Database failed

Tywyn · Feb 6, 2020, 4:56 AM

Hi everyone,
I want to set up a new fog-Server in our new network environment, but since the dhcp-server is being administered by someone else and so I am not sure, how to configure fog. This is what we got:

10.203.128.1: dhcp (administered by someone else - not modifiable)
10.203.128.2: dns
10.203.128.9: fog + dnsmasq

I configured dnsmasq according to this: https://wiki.fogproject.org/wiki/index.php?title=ProxyDHCP_with_dnsmasq

Then I ran the install-script and configured the following:

Here are the settings FOG will use:
Base Linux: Debian
Detected Linux Distribution: Ubuntu
Interface: ens160
Server IP Address: 10.203.128.9
Server Subnet Mask: 255.255.254.0
Server Hostname: fog03
Installation Type: Normal Server
Internationalization: 0
Image Storage Location: /images
Using FOG DHCP: No
DHCP will NOT be setup but you must setup your
| current DHCP server to use FOG for PXE services.

Question 1): Are those the correct settings?
Question 2): Install script says:

Changing permissions on apache log files…OK
Backing up database…OK
Updating Database…Failed!

Anything I can do here?

Sebastian Roth · Feb 6, 2020, 12:21 PM

@Tywyn said:

Interface: ens160
Server IP Address: 10.203.128.9
Server Subnet Mask: 255.255.254.0

Just to make sure as a very first step. Is this IP address already configured for interface ens160 on the Linux system itself? This is not something FOG will do for you! As well pay close attention to the subnet mask. While it is valid subnetting I just want to point to this before we go ahead. Make sure it’s correct.

Updating Database…Failed!

This can only happen if you run the installer with command line parameters. While this is fine I need to ask you which parameters did you use. Post the exact command you used here.

As well, post more details on the Ubuntu version and FOG version you try to use.

Tywyn · Feb 6, 2020, 7:00 AM

@Sebastian-Roth, thanx for your answer.

@Sebastian-Roth said in Unmodifiable DHCP-Server + Fog / Updating Database failed:

@Tywyn said:

Interface: ens160
Server IP Address: 10.203.128.9
Server Subnet Mask: 255.255.254.0

Just to make sure as a very first step. Is this IP address already configured for interface ens160 on the Linux system itself?

Yes it is. subnetmask is correct, as well.

Updating Database…Failed!

This can only happen if you run the installer with command line parameters. While this is fine I need to ask you which parameters did you use. Post the exact command you used here.

As well, post more details on the Ubuntu version and FOG version you try to use.

Ubuntu 18.04.4 LTS
Fog is 1.5.7. I did a git pull and it said, it is up to date.

First time I ran the install-script:

          2) Debian Based Linux (Debian, Ubuntu, Kubuntu, Edubuntu)
          3) Arch Linux

  Choice: [2] 



  Starting Debian based Installation


 *** Detected a potential need to reinstall apache and php files.
 *** This will remove the /etc/php* and /etc/apache2* directories
 ***  and remove/purge the apache and php files from this system.
 *** If you're okay with this please type Y, anything else will
 ***  continue the installation, but may mean you will need to
 ***  remove the files later and make proper changes as 
 ***  necessary. (Y/N): 
y
 * Removing apache and php files...............................Done
 * Stopping web services.......................................Done
 * Removing the apache and php packages........................Done
 * Resetting our variables to specify php version..............Done
  FOG Server installation modes:
      * Normal Server: (Choice N) 
          This is the typical installation type and
          will install all FOG components for you on this
          machine.  Pick this option if you are unsure what to pick.

      * Storage Node: (Choice S)
          This install mode will only install the software required
          to make this server act as a node in a storage group

  More information:  
     http://www.fogproject.org/wiki/index.php?title=InstallationModes

  What type of installation would you like to do? [N/s (Normal/Storage)] 

  We found the following interfaces on your system:
      * ens160 - 10.203.128.9/23

  Would you like to change the default network interface from ens160?
  If you are not sure, select No. [y/N] 

  Would you like to setup a router address for the DHCP server? [Y/n] 
  What is the IP address to be used for the router on
 




      the DHCP server? [10.203.128.1]

  Would you like DHCP to handle DNS? [Y/n] n

  Would you like to use the FOG server for DHCP service? [y/N] 

  This version of FOG has internationalization support, would  
  you like to install the additional language packs? [y/N] 

 *** New option 'hostname' since FOG 1.5.6 ***

  Would you like to change the default hostname >fog03<?
  The fully qualified hostname is used for the webserver certificate.
  If you are not sure, select No. [y/N] 

   ######################################################################
   #     FOG now has everything it needs for this setup, but please     #
   #   understand that this script will overwrite any setting you may   #
   #   have setup for services like DHCP, apache, pxe, tftp, and NFS.   #
   ######################################################################
   # It is not recommended that you install this on a production system #
   #        as this script modifies many of your system settings.       #
   ######################################################################
   #             This script should be run by the root user.            #
   #      It will prepend the running with sudo if root is not set      #
   ######################################################################
   #            Please see our wiki for more information at:            #
   ######################################################################
   #             https://wiki.fogproject.org/wiki/index.php             #
   ######################################################################

 * Here are the settings FOG will use:
 * Base Linux: Debian
 * Detected Linux Distribution: Ubuntu
 * Interface: ens160
 * Server IP Address: 10.203.128.9
 * Server Subnet Mask: 255.255.254.0
 * Server Hostname: fog03
 * Installation Type: Normal Server
 * Internationalization: 0
 * Image Storage Location: /images
 * Using FOG DHCP: No
 * DHCP will NOT be setup but you must setup your
 | current DHCP server to use FOG for PXE services.

 * On a Linux DHCP server you must set: next-server and filename

 * On a Windows DHCP server you must set options 066 and 067

 * Option 066/next-server is the IP of the FOG Server: (e.g. 10.203.128.9)
 * Option 067/filename is the bootfile: (e.g. undionly.kpxe)


 * Are you sure you wish to continue (Y/N) Y

 * Installation Started

Then I noticed that installation of mariadb did not succeed. Reason was obviously, because I installed mysql-server before, because I did not know, that fog now installs everything by itself.

So I did an apt remove mysql-server + apt remove mysql-client and started the fog-install-script with the -y commandline switch.
./installfog.sh -y

Installation of mariadb and client went well then.

Sebastian Roth · Feb 6, 2020, 2:03 PM

@Tywyn Yes, FOG does install all the components needed. It’s been like that since the beginning as far as I know. Good you choose to use MariaDB straight away as this is the default now and while moving from MySQL to MariaDB shouldn’t be too much of trouble Ubuntu is doing a really bad job on this. Sorry to say this but other distros do it way better. Never mind.

Our daily silent install testing (using ./installfog.sh -y as well) reports successful install for Ubuntu 18.04 and FOG 1.5.7 (currently master). So we should probably look at all the logs to see why the step “Updating Database” failed. See my signature on apache and php-fpm logs. As well you have a sub directory in the path where you run the installer script (fogproject/bin/error_logs). Please post the last 20 lines of fog_error_1.5.7.log here.

Tywyn · Feb 6, 2020, 2:10 PM

@Sebastian-Roth

/var/log/apache2/error.log

[Thu Feb 06 11:11:17.955191 2020] [mpm_event:notice] [pid 21469:tid 139950900485056] AH00489: Apache/2.4.29 (Ubuntu) configured -- resuming normal operations
[Thu Feb 06 11:11:17.955301 2020] [core:notice] [pid 21469:tid 139950900485056] AH00094: Command line: '/usr/sbin/apache2'
[Thu Feb 06 11:11:34.605400 2020] [mpm_event:notice] [pid 21469:tid 139950900485056] AH00491: caught SIGTERM, shutting down
[Thu Feb 06 11:11:34.679030 2020] [mpm_prefork:notice] [pid 27475] AH00163: Apache/2.4.29 (Ubuntu) configured -- resuming normal operations
[Thu Feb 06 11:11:34.679133 2020] [core:notice] [pid 27475] AH00094: Command line: '/usr/sbin/apache2'
[Thu Feb 06 11:11:35.117284 2020] [mpm_prefork:notice] [pid 27475] AH00169: caught SIGTERM, shutting down
[Thu Feb 06 11:11:35.207674 2020] [mpm_prefork:notice] [pid 27576] AH00163: Apache/2.4.29 (Ubuntu) configured -- resuming normal operations
[Thu Feb 06 11:11:35.208024 2020] [core:notice] [pid 27576] AH00094: Command line: '/usr/sbin/apache2'
[Thu Feb 06 11:15:30.006867 2020] [mpm_prefork:notice] [pid 27576] AH00169: caught SIGTERM, shutting down
[Thu Feb 06 11:15:39.984979 2020] [mpm_prefork:notice] [pid 2080] AH00163: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.1 configured -- resuming normal operations
[Thu Feb 06 11:15:39.985048 2020] [core:notice] [pid 2080] AH00094: Command line: '/usr/sbin/apache2'

/var/log/php7.2-fpm.log

[06-Feb-2020 10:58:41] NOTICE: fpm is running, pid 15352
[06-Feb-2020 10:58:41] NOTICE: ready to handle connections
[06-Feb-2020 10:58:41] NOTICE: systemd monitor interval set to 10000ms
[06-Feb-2020 11:10:45] NOTICE: Terminating ...
[06-Feb-2020 11:10:45] NOTICE: exiting, bye-bye!
[06-Feb-2020 11:12:04] NOTICE: fpm is running, pid 31676
[06-Feb-2020 11:12:04] NOTICE: ready to handle connections
[06-Feb-2020 11:12:04] NOTICE: systemd monitor interval set to 10000ms
[06-Feb-2020 11:12:08] NOTICE: Terminating ...
[06-Feb-2020 11:12:08] NOTICE: exiting, bye-bye!
[06-Feb-2020 11:12:08] NOTICE: fpm is running, pid 32139
[06-Feb-2020 11:12:08] NOTICE: ready to handle connections
[06-Feb-2020 11:12:08] NOTICE: systemd monitor interval set to 10000ms
[06-Feb-2020 11:12:12] NOTICE: Terminating ...
[06-Feb-2020 11:12:12] NOTICE: exiting, bye-bye!
[06-Feb-2020 11:12:12] NOTICE: fpm is running, pid 32563
[06-Feb-2020 11:12:12] NOTICE: ready to handle connections
[06-Feb-2020 11:12:12] NOTICE: systemd monitor interval set to 10000ms
[06-Feb-2020 11:12:17] NOTICE: Terminating ...
[06-Feb-2020 11:12:17] NOTICE: exiting, bye-bye!
[06-Feb-2020 11:12:17] NOTICE: fpm is running, pid 1073
[06-Feb-2020 11:12:17] NOTICE: ready to handle connections
[06-Feb-2020 11:12:17] NOTICE: systemd monitor interval set to 10000ms
[06-Feb-2020 11:15:29] NOTICE: Terminating ...
[06-Feb-2020 11:15:29] NOTICE: exiting, bye-bye!
[06-Feb-2020 11:15:39] NOTICE: fpm is running, pid 2068
[06-Feb-2020 11:15:39] NOTICE: ready to handle connections
[06-Feb-2020 11:15:39] NOTICE: systemd monitor interval set to 10000ms

error log (some more than just 20 lines)

ii  php7.2-gd      7.2.24-0ubuntu0.18.04.2 amd64        GD module for PHP
ii  php7.2-json    7.2.24-0ubuntu0.18.04.2 amd64        JSON module for PHP
ii  php7.2-ldap    7.2.24-0ubuntu0.18.04.2 amd64        LDAP module for PHP
ii  php7.2-mbstring 7.2.24-0ubuntu0.18.04.2 amd64        MBSTRING module for PHP
ii  php7.2-mysql   7.2.24-0ubuntu0.18.04.2 amd64        MySQL module for PHP
ii  tar            1.29b-2ubuntu0.1 amd64        GNU version of the tar archiving utility
ii  tftp-hpa       5.2+20150808-1ubuntu3 amd64        HPA's tftp client
ii  tftpd-hpa      5.2+20150808-1ubuntu3 amd64        HPA's tftp server
ii  unzip          6.0-21ubuntu1 amd64        De-archiver for .zip files
ii  vsftpd         3.0.3-9build1 amd64        lightweight, efficient FTP server written for security
ii  wget           1.19.4-1ubuntu2.2 amd64        retrieves files from the web
ii  xinetd         1:2.3.15.3-1 amd64        replacement for inetd with many enhancements
ii  zlib1g:amd64   1:1.2.11.dfsg-0ubuntu2 amd64        compression library - runtime
useradd: warning: the home directory already exists.
Not copying any file from skel directory into it.
Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully
Job for mariadb.service failed because a timeout was exceeded.
See "systemctl status mariadb.service" and "journalctl -xe" for details.
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2 "No such file or directory")
Job for mariadb.service failed because a timeout was exceeded.
See "systemctl status mariadb.service" and "journalctl -xe" for details.
find: ‘/home//fog_web_1.5.7.BACKUP/management/other/’: No such file or directory
binaries1.5.7.zip: OK
Archive:  binaries1.5.7.zip
  inflating: packages/kernels/bzImage  
  inflating: packages/kernels/bzImage32  
  inflating: packages/kernels/arm_Image32  
  inflating: packages/kernels/arm_Image  
  inflating: packages/inits/arm_init_32.cpio.gz  
  inflating: packages/inits/arm_init.cpio.gz  
  inflating: packages/inits/init_32.xz  
  inflating: packages/inits/init.xz  
  inflating: packages/clientfiles/SmartInstaller.exe  
  inflating: packages/clientfiles/FOGService.msi  
'packages/kernels/bzImage' -> '/var/www/html/fog//service/ipxe/bzImage'
'packages/kernels/bzImage32' -> '/var/www/html/fog//service/ipxe/bzImage32'
'packages/inits/init.xz' -> '/var/www/html/fog//service/ipxe/init.xz'
'packages/inits/init_32.xz' -> '/var/www/html/fog//service/ipxe/init_32.xz'
'packages/clientfiles/FOGService.msi' -> '/var/www/html/fog//client/FOGService.msi'
'packages/clientfiles/SmartInstaller.exe' -> '/var/www/html/fog//client/SmartInstaller.exe'
mysqlnd
Synchronizing state of apache2.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable apache2
Synchronizing state of php7.2-fpm.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable php7.2-fpm
Generating RSA private key, 4096 bit long modulus (2 primes)
................................++++
.............................................................................++++
e is 65537 (0x010001)
Can't load /root/.rnd into RNG
140342283981248:error:2406F079:random number generator:RAND_load_file:Cannot open file:../crypto/rand/randfile.c:88:Filename=/root/.rnd
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:State or Province Name (full name) [Some-State]:Locality Name (eg, city) []:Organization Name (eg, company) [Internet Widgits Pty Ltd]:Organizational Unit Name (eg, section) []:Common Name (e.g. server FQDN or YOUR name) []:Email Address []:Generating RSA private key, 4096 bit long modulus (2 primes)
............................................................++++
.........................................................................................................................++++
e is 65537 (0x010001)
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
10.203.128.9 []:Signature ok
subject=CN = 10.203.128.9
Getting CA Private Key
ln: failed to create symbolic link '/var/www/html/fog/fog': File exists
ERROR: Module php does not exist!
Considering dependency proxy for proxy_fcgi:
Enabling module proxy.
Enabling module proxy_fcgi.
Module setenvif already enabled
To activate the new configuration, you need to run:
  systemctl restart apache2
Enabling module rewrite.
To activate the new configuration, you need to run:
  systemctl restart apache2
Considering dependency setenvif for ssl:
Module setenvif already enabled
Considering dependency mime for ssl:
Module mime already enabled
Considering dependency socache_shmcb for ssl:
Enabling module socache_shmcb.
Enabling module ssl.
See /usr/share/doc/apache2/README.Debian.gz on how to configure SSL and create self-signed certificates.
To activate the new configuration, you need to run:
  systemctl restart apache2
Enabling site 001-fog.
To activate the new configuration, you need to run:
  systemctl reload apache2
Site 000-default disabled.
To activate the new configuration, you need to run:
  systemctl reload apache2
● apache2.service - The Apache HTTP Server
   Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/apache2.service.d
           └─apache2-systemd.conf
   Active: active (running) since Thu 2020-02-06 11:15:39 UTC; 2s ago
  Process: 2066 ExecStart=/usr/sbin/apachectl start (code=exited, status=0/SUCCESS)
 Main PID: 2080 (apache2)
    Tasks: 6 (limit: 4660)
   CGroup: /system.slice/apache2.service
           ├─2080 /usr/sbin/apache2 -k start
           ├─2098 /usr/sbin/apache2 -k start
           ├─2099 /usr/sbin/apache2 -k start
           ├─2100 /usr/sbin/apache2 -k start
           ├─2102 /usr/sbin/apache2 -k start
           └─2105 /usr/sbin/apache2 -k start

Feb 06 11:15:39 fog03 systemd[1]: Starting The Apache HTTP Server...
Feb 06 11:15:39 fog03 systemd[1]: Started The Apache HTTP Server.

● php7.2-fpm.service - The PHP 7.2 FastCGI Process Manager
   Loaded: loaded (/lib/systemd/system/php7.2-fpm.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2020-02-06 11:15:39 UTC; 2s ago
     Docs: man:php-fpm7.2(8)
 Main PID: 2068 (php-fpm7.2)
   Status: "Ready to handle connections"
    Tasks: 6 (limit: 4660)
   CGroup: /system.slice/php7.2-fpm.service
           ├─2068 php-fpm: master process (/etc/php/7.2/fpm/php-fpm.conf)
           ├─2081 php-fpm: pool www
           ├─2083 php-fpm: pool www
           ├─2084 php-fpm: pool www
           ├─2085 php-fpm: pool www
           └─2087 php-fpm: pool www

Feb 06 11:15:39 fog03 systemd[1]: Starting The PHP 7.2 FastCGI Process Manager...
Feb 06 11:15:39 fog03 systemd[1]: Started The PHP 7.2 FastCGI Process Manager.

Sebastian Roth · Feb 6, 2020, 5:52 PM

@Tywyn said:

Job for mariadb.service failed because a timeout was exceeded.
See "systemctl status mariadb.service" and "journalctl -xe" for details.
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2 "No such file or directory")
Job for mariadb.service failed because a timeout was exceeded.
See "systemctl status mariadb.service" and "journalctl -xe" for details.
find: ‘/home//fog_web_1.5.7.BACKUP/management/other/’: No such file or directory

Should have thought about this earlier. There is something wrong with your database. Did you install MariaDB manually beforehand? Is there any other application running on this server using MariaDB?

Please run systemctl status mariadb.service and journalctl -xe. Post both outputs here.

Tywyn · Feb 10, 2020, 2:57 AM

@Sebastian-Roth said in Unmodifiable DHCP-Server + Fog / Updating Database failed:

@Tywyn said:

See “systemctl status mariadb.service” and “journalctl -xe” for details.
find: ‘/home//fog_web_1.5.7.BACKUP/management/other/’: No such file or directory

Should have thought about this earlier. There is something wrong with your database. Did you install MariaDB manually beforehand? Is there any other application running on this server using MariaDB?

Please run systemctl status mariadb.service and journalctl -xe. Post both outputs here.

I installed mysql manually beforehand, but since I realized, that fog comes with its own mariadb-installation, I removed the packages mysql-server and mysql-client.
I see, there is still mysql-common installed, but it seems, that mariadb needs this package, or?

I did not install any other applications on the server which use a database. At least I don’t know about any.

# systemctl status mariadb.service
● mariadb.service - MariaDB 10.1.44 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
   Active: failed (Result: timeout) since Mon 2020-02-10 08:32:41 UTC; 15min ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 26796 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=0/SUCCESS)
  Process: 26722 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (co
  Process: 26714 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
  Process: 26703 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
 Main PID: 26796 (code=exited, status=0/SUCCESS)

Feb 10 08:17:38 fog03 systemd[1]: Starting MariaDB 10.1.44 database server...
Feb 10 08:17:39 fog03 mysqld[26796]: 2020-02-10  8:17:39 140150343179392 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 26796 ...
Feb 10 08:32:39 fog03 systemd[1]: mariadb.service: Start operation timed out. Terminating.
Feb 10 08:32:41 fog03 systemd[1]: mariadb.service: Failed with result 'timeout'.
Feb 10 08:32:41 fog03 systemd[1]: Failed to start MariaDB 10.1.44 database server.

# journalctl -xe
Feb 10 08:09:08 fog03 systemd[1]: Started Clean php session files.
-- Subject: Unit phpsessionclean.service has finished start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit phpsessionclean.service has finished starting up.
-- 
-- The start-up result is RESULT.
Feb 10 08:17:01 fog03 CRON[26696]: pam_unix(cron:session): session opened for user root by (uid=0)
Feb 10 08:17:01 fog03 CRON[26697]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Feb 10 08:17:01 fog03 CRON[26696]: pam_unix(cron:session): session closed for user root
Feb 10 08:17:38 fog03 systemd[1]: Starting MariaDB 10.1.44 database server...
-- Subject: Unit mariadb.service has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit mariadb.service has begun starting up.
Feb 10 08:17:39 fog03 mysqld[26796]: 2020-02-10  8:17:39 140150343179392 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 26796 ...
Feb 10 08:17:39 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:17:39 fog03 kernel: kauditd_printk_skb: 2 callbacks suppressed
Feb 10 08:17:39 fog03 kernel: audit: type=1400 audit(1581322659.769:90): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/
Feb 10 08:32:39 fog03 systemd[1]: mariadb.service: Start operation timed out. Terminating.
Feb 10 08:32:39 fog03 kernel: audit: type=1400 audit(1581323559.325:91): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/
Feb 10 08:32:39 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:39 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:39 fog03 kernel: audit: type=1400 audit(1581323559.429:92): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/
Feb 10 08:32:39 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:39 fog03 kernel: audit: type=1400 audit(1581323559.529:93): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/
Feb 10 08:32:39 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:39 fog03 kernel: audit: type=1400 audit(1581323559.629:94): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/
Feb 10 08:32:39 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:39 fog03 kernel: audit: type=1400 audit(1581323559.729:95): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/
Feb 10 08:32:39 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:39 fog03 kernel: audit: type=1400 audit(1581323559.829:96): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/
Feb 10 08:32:39 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:39 fog03 kernel: audit: type=1400 audit(1581323559.929:97): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/
Feb 10 08:32:40 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:40 fog03 kernel: audit: type=1400 audit(1581323560.029:98): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/
Feb 10 08:32:40 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:40 fog03 kernel: audit: type=1400 audit(1581323560.129:99): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/
Feb 10 08:32:40 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:40 fog03 kernel: audit: type=1400 audit(1581323560.233:100): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run
Feb 10 08:32:40 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:40 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:40 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:40 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:41 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:41 fog03 audit[26796]: AVC apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=26796 comm="m
Feb 10 08:32:41 fog03 systemd[1]: mariadb.service: Failed with result 'timeout'.
Feb 10 08:32:41 fog03 systemd[1]: Failed to start MariaDB 10.1.44 database server.
-- Subject: Unit mariadb.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit mariadb.service has failed.
-- 
-- The result is RESULT.
Feb 10 08:39:01 fog03 CRON[26848]: pam_unix(cron:session): session opened for user root by (uid=0)
Feb 10 08:39:01 fog03 CRON[26849]: (root) CMD (  [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
Feb 10 08:39:01 fog03 CRON[26848]: pam_unix(cron:session): session closed for user root
Feb 10 08:39:08 fog03 systemd[1]: Starting Clean php session files...
-- Subject: Unit phpsessionclean.service has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit phpsessionclean.service has begun starting up.
Feb 10 08:39:08 fog03 systemd[1]: Started Clean php session files.
-- Subject: Unit phpsessionclean.service has finished start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit phpsessionclean.service has finished starting up.
-- 
-- The start-up result is RESULT.

Sebastian Roth · Feb 10, 2020, 6:03 AM

@Tywyn said in Unmodifiable DHCP-Server + Fog / Updating Database failed:

I did not install any other applications on the server which use a database. At least I don’t know about any.

First you should double check there is not something in the DB that you might loose. Run ls -al /var/lib/mysql and post output here.

If you are absolutely sure there is nothing we could loose in the database I would say you better uninstall all the database packages (apt-get remove mysql* mariadb*), move the old DB files out of the way (mv /var/lib/mysql /var/lib/mysql_old) and re-run the FOG installer without -y switch.

Tywyn · Feb 10, 2020, 6:52 AM

@Sebastian-Roth
Removed all mysql-common and mariabd* Packages.
Reran the installer. It took about 15 minutes to complete this line
* Installing package: mariadb-server..........................OK

Then it complained about an existing user “fogproject”. So I removed it and restarted the installer.

Now I am waiting for about 15 Minutes to complete this:
* Setting up and starting MySQL...............................

Does it usually take that long?

I found in the syslog:

Feb 10 12:46:48 fog03 kernel: [356864.470279] audit: type=1400 audit(1581338808.728:131): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=31939 comm="mysqld" requested_mask="w" denied_mask="w" fsuid=111 ouid=0
Feb 10 12:46:50 fog03 systemd[1]: mariadb.service: Failed with result 'timeout'.
Feb 10 12:46:50 fog03 systemd[1]: Failed to start MariaDB 10.1.44 database server.
Feb 10 12:46:50 fog03 systemd[1]: Starting MariaDB 10.1.44 database server...
Feb 10 12:46:51 fog03 mysqld[32200]: 2020-02-10 12:46:51 140481301748864 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 32200 ...

Very strange … I guess I will setup the machine from scratch …

So, just to be sure: The steps are:

Install OS … then

apt-get -y install git
cd /root
git clone https://github.com/FOGProject/fogproject.git
./installfog.sh

And nothing more, right?

Tywyn · Feb 10, 2020, 7:37 AM

@Sebastian-Roth

ok, I gave it another try with the system.
I removed the package apparmor and ran the installer again (no commandline flags)

Last line was:
* Backing up database.........................................Failed!

fog - error log:

   Loaded: loaded (/lib/systemd/system/php7.2-fpm.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-02-10 13:30:46 UTC; 6s ago
     Docs: man:php-fpm7.2(8)
 Main PID: 19674 (php-fpm7.2)
   Status: "Ready to handle connections"
    Tasks: 6 (limit: 4660)
   CGroup: /system.slice/php7.2-fpm.service
           ├─19674 php-fpm: master process (/etc/php/7.2/fpm/php-fpm.conf)
           ├─19691 php-fpm: pool www
           ├─19692 php-fpm: pool www
           ├─19693 php-fpm: pool www
           ├─19694 php-fpm: pool www
           └─19695 php-fpm: pool www

Feb 10 13:30:45 fog03 systemd[1]: Starting The PHP 7.2 FastCGI Process Manager...
Feb 10 13:30:46 fog03 systemd[1]: Started The PHP 7.2 FastCGI Process Manager.
--2020-02-10 13:30:53--  http://10.203.128.9//fog//maintenance/backup_db.php
Connecting to 10.203.128.9:80... connected.
HTTP request sent, awaiting response... 302 Found
Could not parse String-Transport-Security header
Location: ?node=schema [following]
--2020-02-10 13:30:53--  http://10.203.128.9//fog//maintenance/backup_db.php?node=schema
Connecting to 10.203.128.9:80... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2020-02-10 13:30:53 ERROR 500: Internal Server Error.

# systemctl status mysql
● mariadb.service - MariaDB 10.1.44 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-02-10 13:30:29 UTC; 4min 12s ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
 Main PID: 19261 (mysqld)
   Status: "Taking your SQL requests now..."
    Tasks: 28 (limit: 4660)
   CGroup: /system.slice/mariadb.service
           └─19261 /usr/sbin/mysqld

Feb 10 13:30:28 fog03 systemd[1]: Starting MariaDB 10.1.44 database server...
Feb 10 13:30:28 fog03 mysqld[19261]: 2020-02-10 13:30:28 140079113030784 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 19261 ...
Feb 10 13:30:29 fog03 /etc/mysql/debian-start[19298]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
Feb 10 13:30:29 fog03 /etc/mysql/debian-start[19298]: Looking for 'mysql' as: /usr/bin/mysql
Feb 10 13:30:29 fog03 /etc/mysql/debian-start[19298]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
Feb 10 13:30:29 fog03 /etc/mysql/debian-start[19298]: This installation of MySQL is already upgraded to 10.1.44-MariaDB, use --force if you still need to run mysql_upgrade
Feb 10 13:30:29 fog03 /etc/mysql/debian-start[19310]: Triggering myisam-recover for all MyISAM tables and aria-recover for all Aria tables
Feb 10 13:30:29 fog03 systemd[1]: Started MariaDB 10.1.44 database server.

# journalctl -xe
Feb 10 13:30:29 fog03 systemd[1]: Started MariaDB 10.1.44 database server.
-- Subject: Unit mariadb.service has finished start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit mariadb.service has finished starting up.
-- 
-- The start-up result is RESULT.
Feb 10 13:30:29 fog03 systemd[1]: Stopping The Apache HTTP Server...
-- Subject: Unit apache2.service has begun shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit apache2.service has begun shutting down.
Feb 10 13:30:29 fog03 systemd[1]: Stopping The PHP 7.2 FastCGI Process Manager...
-- Subject: Unit php7.2-fpm.service has begun shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit php7.2-fpm.service has begun shutting down.
Feb 10 13:30:29 fog03 systemd[1]: Stopped The PHP 7.2 FastCGI Process Manager.
-- Subject: Unit php7.2-fpm.service has finished shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit php7.2-fpm.service has finished shutting down.
Feb 10 13:30:29 fog03 systemd[1]: Stopped The Apache HTTP Server.
-- Subject: Unit apache2.service has finished shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit apache2.service has finished shutting down.
Feb 10 13:30:39 fog03 systemd[1]: Reloading.
Feb 10 13:30:40 fog03 systemd[1]: Reloading.
Feb 10 13:30:40 fog03 systemd[1]: Reloading.
Feb 10 13:30:40 fog03 systemd[1]: Reloading.
Feb 10 13:30:40 fog03 systemd[1]: Reloading.
Feb 10 13:30:41 fog03 systemd[1]: Reloading.
Feb 10 13:30:45 fog03 systemd[1]: Starting The Apache HTTP Server...
-- Subject: Unit apache2.service has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit apache2.service has begun starting up.
Feb 10 13:30:45 fog03 systemd[1]: Starting The PHP 7.2 FastCGI Process Manager...
-- Subject: Unit php7.2-fpm.service has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit php7.2-fpm.service has begun starting up.
Feb 10 13:30:46 fog03 systemd[1]: Started The PHP 7.2 FastCGI Process Manager.
-- Subject: Unit php7.2-fpm.service has finished start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit php7.2-fpm.service has finished starting up.
-- 
-- The start-up result is RESULT.
Feb 10 13:30:46 fog03 systemd[1]: Started The Apache HTTP Server.
-- Subject: Unit apache2.service has finished start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit apache2.service has finished starting up.
-- 
-- The start-up result is RESULT.

Tywyn · Feb 10, 2020, 4:25 PM

And here is the apache-error log:

[Mon Feb 10 13:29:44.474607 2020] [mpm_prefork:notice] [pid 13274] AH00169: caught SIGTERM, shutting down
[Mon Feb 10 13:29:44.550774 2020] [mpm_prefork:notice] [pid 13369] AH00163: Apache/2.4.29 (Ubuntu) configured -- resuming normal operations
[Mon Feb 10 13:29:44.550826 2020] [core:notice] [pid 13369] AH00094: Command line: '/usr/sbin/apache2'
[Mon Feb 10 13:30:29.575791 2020] [mpm_prefork:notice] [pid 13369] AH00169: caught SIGTERM, shutting down
[Mon Feb 10 13:30:46.715203 2020] [mpm_prefork:notice] [pid 19708] AH00163: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.1 configured -- resuming normal operations
[Mon Feb 10 13:30:46.715263 2020] [core:notice] [pid 19708] AH00094: Command line: '/usr/sbin/apache2'
[Mon Feb 10 13:30:53.692560 2020] [proxy_fcgi:error] [pid 19715] [client 10.203.128.9:34104] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: Connection to mysql failed with message: SQLSTATE[HY000] [1049] Unknown database 'fog' in /var/www/html/fog/lib/db/mysqldump.class.php:282\nStack trace:\n#0 /var/www/html/fog/lib/db/mysqldump.class.php(306): Mysqldump->connect()\n#1 /var/www/html/fog/lib/fog/schema.class.php(200): Mysqldump->start('/tmp/fog_backup...')\n#2 /var/www/html/fog/lib/fog/reportmaker.class.php(269): Schema->exportdb('fog_backup_2020...')\n#3 /var/www/html/fog/maintenance/backup_db.php(23): ReportMaker->outputReport(3, true)\n#4 {main}\n  thrown in /var/www/html/fog/lib/db/mysqldump.class.php on line 282\n'

george1421 · Feb 10, 2020, 4:33 PM

@Tywyn So that error is saying that it can’t find the fog database on the server.

Hopefully you know the root password for the database. You should have been asked that question when you first setup FOG. Originally that password was blank, but later installers prompted the FOG Installer (person) to create a password.

If your mysql database root user doesn’t have a password key in this.

mysql -u root

If you are presented with an sql server command prompt then go to the next section.

If your mysql server requires a root password key in this.

mysql -u root -p

And then enter root’s password

Once you are at the mysql command prompt key in : show databases;
The response should look like this

MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| fog                |
| mysql              |
| performance_schema |
| test               |
+--------------------+
5 rows in set (0.10 sec)

I’m suspecting that the fog database is missing. Rerunning the installer should recreate that database for you. During the installer it will pause and prompt you to go to the FOG web ui and install the schema. Once the schema is installed return to the fog installer and continue the setup.

Sebastian Roth · Feb 10, 2020, 7:13 PM

@Tywyn said in Unmodifiable DHCP-Server + Fog / Updating Database failed:

Very strange …

Well, seems like something is really wrong there. I do remember having some strange issue with the DB on Ubuntu myself at some point and through it might be related to the apparmor messages. But as far as I remember this wasn’t actually causing the problem as they were all just warnings. Unfortunately I don’t remember what exactly it was. Maybe you can find something in the DB server logs? Should be in /var/log/mysql or /var/log/mariadb I think.

I guess I will setup the machine from scratch …

While you will learn a lot by digging through this things and finding out what is causing it and how to fix it I also side with the idea of just setting it up from scratch.

So, just to be sure: The steps are:
Install OS … then
apt-get -y install git
cd /root
git clone https://github.com/FOGProject/fogproject.git
./installfog.sh

Pretty much. Just one command is missing:

...
git clone https://github.com/FOGProject/fogproject
cd fogproject/bin/
./installfog.sh

Tywyn · Feb 11, 2020, 9:39 AM

@Sebastian-Roth , @george1421

Thanx for your help.
I set up a new virtual host with ubuntu 18.04.03 LTS server with fog and dnsmasq and everything went fine for the installation part now.
The ubuntu 18.04.03 provides apparmor by default, so I assume, that apparmor was not the reason for the install-issues I had before.
Maybe the fact, that I have had mysql installed manually, tried installation of fog and removed mysql after noticing, that fog wants to install mariadb, gave us the troubles we had.

Now another problem arose, but I opened another thread for that: https://forums.fogproject.org/topic/14197/1-5-7-does-not-capture-correctly

Unmodifiable DHCP-Server + Fog / Updating Database failed

183

12.0k

17.3k

155.1k