No startable FOG processes and empty DB backups after FOG upgrade
-
@Tom-Elliott OK, this is pretty much what I was expecting. So I’ve tried those steps and it looks good up until the point where installfog.sh fails at “Stopping web service”. Here is the foginstall.log which ends with:
0_1479061190246_foginstall.log* Setting up fog user.........................................Already setup * Setting up fog password.....................................OK * Stopping FOGMulticastManager Service........................OK * Stopping FOGImageReplicator Service.........................OK * Stopping FOGSnapinReplicator Service........................OK * Stopping FOGScheduler Service...............................OK * Stopping FOGPingHosts Service...............................OK * Setting up and starting MySQL...............................OK * Backing up user reports.....................................Done * Stopping web service........................................Failed!Here is the fog_error_1.3.0-RC-22.log which ends with:
0_1479061317524_fog_error_1.3.0-RC-22.log* Stopping FOG Computer Imaging Solution: FOGScheduler ...done. * Stopping FOG Computer Imaging Solution: FOGPingHosts ...done. mysql stop/waiting mysql start/running, process 25774 /etc/init.d/apache2: 64: .: Can't open /etc/apache2/envvars /etc/init.d/apache2: 76: .: Can't open /etc/apache2/envvars ERROR: APACHE_PID_FILE needs to be defined in /etc/apache2/envvars/etc/apache2/envvars does not exist. Here is a listing of the /etc/apache2 directory:
root@FogU1404T01:/etc/apache2# lsa total 24 drwxr-xr-x 4 root root 4096 Nov 13 09:53 . drwxr-xr-x 141 root root 12288 Nov 13 10:04 .. drwxr-xr-x 2 root root 4096 Nov 13 09:54 conf-available drwxr-xr-x 2 root root 4096 Nov 13 09:51 mods-available root@FogU1404T01:/etc/apache2#I think this may be caused by old php5 processes still running as shown here:
root@FogU1404T01:~# ps -ef | grep php root 1717 1 0 09:26 ? 00:00:00 php-fpm: master process (/etc/php5/fpm/php-fpm.conf) www-data 1719 1717 0 09:26 ? 00:00:00 php-fpm: pool www www-data 1720 1717 0 09:26 ? 00:00:00 php-fpm: pool www root 25547 1 0 10:04 ? 00:00:00 php-fpm: master process (/etc/php/7.0/fpm/php-fpm.conf) www-data 25550 25547 0 10:04 ? 00:00:00 php-fpm: pool www www-data 25551 25547 0 10:04 ? 00:00:00 php-fpm: pool www root 26194 3422 0 10:28 pts/4 00:00:00 grep --color=auto phpI could restore my snapshot and try again after making sure there are no php5 processes running. What do you think?
-
After rebooting the server I now have the FOG processes and the new php7 processes running as shown here.
root@FogU1404T01:~# ps -ef | grep php root 1146 1 0 10:59 ? 00:00:00 php-fpm: master process (/etc/php/7.0/fpm/php-fpm.conf) www-data 1557 1146 0 10:59 ? 00:00:00 php-fpm: pool www www-data 1558 1146 0 10:59 ? 00:00:00 php-fpm: pool www root 1576 1 0 10:59 ? 00:00:00 /usr/bin/php -q /opt/fog/service/FOGSnapinReplicator/FOGSnapinReplicator root 1600 1 0 10:59 ? 00:00:00 /usr/bin/php -q /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler root 1623 1 0 10:59 ? 00:00:00 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager root 1646 1 0 10:59 ? 00:00:00 /usr/bin/php -q /opt/fog/service/FOGImageReplicator/FOGImageReplicator root 1670 1 0 10:59 ? 00:00:00 /usr/bin/php -q /opt/fog/service/FOGPingHosts/FOGPingHosts root 2167 1600 0 10:59 ? 00:00:00 /usr/bin/php -q /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler root 2168 1646 0 10:59 ? 00:00:00 /usr/bin/php -q /opt/fog/service/FOGImageReplicator/FOGImageReplicator root 2169 1670 0 10:59 ? 00:00:00 /usr/bin/php -q /opt/fog/service/FOGPingHosts/FOGPingHosts root 2170 1576 0 10:59 ? 00:00:00 /usr/bin/php -q /opt/fog/service/FOGSnapinReplicator/FOGSnapinReplicator root 2171 1623 0 10:59 ? 00:00:00 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager root 3570 3526 0 11:01 pts/4 00:00:00 grep --color=auto phpBut I don’t have any web GUI, obviously since the installfog.sh script never completed. So I’m going to first try rerunning the installer. And if that doesn’t work, I’ll restore my snapshot and try the steps again - this time making sure I don’t have any old php5 processes running.
-
@Tom-Elliott Rerunning the installer fails with the same results as before. Restoring the snapshot and trying the steps again after killing the three old php5 running processes also fails with the same results. Here are the logs from the last attempt with no php5 processes running.
1_1479073227092_foginstall.log
0_1479073227091_fog_error_1.3.0-RC-22.logAs with the other attempts, the directory /etc/apache2 only contains the subdirectories conf-available and mods-available. /etc/apache2/envvars does not exist.
-
@utopia can you run:
a2dismod envvars -
root@FogU1404T01:~# a2dismod envvars ERROR: Module envvars does not exist! root@FogU1404T01:~# -
-
@Tom-Elliott I suppose because it’s stopping this is the issue.
Maybe try:
touch /etc/apache2/envvars -
root@FogU1404T01:~# rm -rf /etc/apache*; apt-get purge apache2 Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libntdb1 python-ntdb Use 'apt-get autoremove' to remove them. The following packages will be REMOVED: apache2* libapache2-mod-php5* 0 upgraded, 0 newly installed, 2 to remove and 3 not upgraded. After this operation, 10.2 MB disk space will be freed. Do you want to continue? [Y/n] y (Reading database ... 173250 files and directories currently installed.) Removing libapache2-mod-php5 (5.6.23+dfsg-1+deprecated+dontuse+deb.sury.org~trusty+1) ... apache2_invoke php5 prerm: No action required Purging configuration files for libapache2-mod-php5 (5.6.23+dfsg-1+deprecated+dontuse+deb.sury.org~trusty+1) ... apache2_invoke php5 postrm: No action required Removing apache2 (2.4.18-1+deb.sury.org~trusty+2) ... /etc/init.d/apache2: 64: .: Can't open /etc/apache2/envvars /etc/init.d/apache2: 76: .: Can't open /etc/apache2/envvars ERROR: APACHE_PID_FILE needs to be defined in /etc/apache2/envvars invoke-rc.d: initscript apache2, action "stop" failed. Purging configuration files for apache2 (2.4.18-1+deb.sury.org~trusty+2) ... dpkg: warning: while removing apache2, directory '/var/www/html' not empty so not removed Processing triggers for man-db (2.6.7.1-1ubuntu1) ... root@FogU1404T01:~#Then reran script
* Adding needed repository....................................OK * Preparing Package Manager...................................OK * Packages to be installed: apache2 bc build-essential cpp curl g++ gawk gcc gzip htmldoc lftp libapache2-mod-php7.0 libc6 libcurl3 m4 mysql-client mysql-server net-tools nfs-kernel-server openssh-server php7.0 php7.0-bcmath php7.0-cli php7.0-curl php7.0-fpm php7.0-gd php7.0-json php7.0-ldap php7.0-mbstring php7.0-mcrypt php7.0-mysqlnd php-gettext sysv-rc-conf tar tftpd-hpa tftp-hpa vsftpd wget xinetd zlib1g * Installing package: apache2.................................Failed!I assumed that I would touch the envvars file after the apache2 package was reinstalled, then run the installer again after that if necessary. But since the package installation fails, I can’t get that far. Do you want the two log files from this?
-
@utopia I suppose you could do the manual installation of apache.
Or it just failed out because of the warning (the /var/www/html issue).
If you need to test it, I suppose you could do:
apt-get install apache2 -f service apache2 start service apache2 stop -
@Tom-Elliott “apt-get install apache2 -f” fails. However, aptitude was able to untangle the dependency trees. For your reference, the attached file shows a rough summary of the steps I took to sort it out.
0_1479102893333_Fog_troubleshooting_161113d.txtBased upon the new php 7.0, I was able to successfully upgrade FOG from RC-18 to RC-22. This gave me running FOG processes and a functional GUI with the new apache2. Then, to test it out further, I upgraded again from RC-22 to RC-23. That successfully showed that I once again get automated DB backup files created during the FOG upgrade process which now do contain data (no longer zero byte size files). There are still some stray php5 processes which try to start and get killed during server bootup - which I will look into cleaning up. And I will be monitoring this server a bit more closely over the next few upgrades to make sure it remains stable.
I do have a final related follow-up question. As the end of the attached file shows, the database backup files have different sizes depending upon whether they are created automatically during the FOG upgrade or created manually via the command line, such as “mysqldump --allow-keywords -x -v fog > fogbackup.sql”. I assume that both types of DB backup are equally useful for restoring or transferring the DB to either the same or a different FOG server - assuming of course that the target server is running the proper FOG version to match the DB backup file. Would that be a correct assumption that either method of creating the DB backup files is equally useful for restore or transfer?
Lastly, I most certainly do THANK YOU for your insight in helping sort this out. Your assistance is VERY MUCH APPRECIATED!
-
@utopia There is some variants in what’s going on for SQL backup. While the version that “automatically” creates a new backup is slightly larger, this is mostly just comments.
I’ve seen some strange things in regards to Ubuntu in the past. The unfortunate part is that not all systems are the same. What you experienced does not mean everybody experiences. The only other thing I can think is ondrej changed how the repo is used relatively recently. Of note, php5-fpm is no longer proper, it’s supposed to be php5.6-fpm as described earlier in the posting. I’ve added this change for the working branch now so hopefully this will no longer be an issue.
I don’t know what php5 items are still lingering, but I’d imagine you could remove them with aptitude just as well?