Well crap. It happened again. I was looking away so missed it:
[04-27-21 7:09:59 am] Interface not ready, waiting for it to come up:
[04-27-21 7:10:09 am] Interface not ready, waiting for it to come up:
[04-27-21 7:10:19 am] Interface not ready, waiting for it to come up:
I think the logs are cleaned up a little too quickly for me to catch what’s going on. Any way to use logrotate on them rather than wiping them out upon service restarts?
@tom-elliott This setting? Haven’t modified it since initial installation:
# df -hPT
Filesystem Type Size Used Avail Use% Mounted on
devtmpfs devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs tmpfs 1.9G 9.0M 1.9G 1% /run
tmpfs tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/mapper/cl_fog-root xfs 16G 6.3G 9.8G 40% /
/dev/sda1 xfs 1014M 312M 703M 31% /boot
/dev/mapper/cl_fog-var xfs 5.0G 3.4G 1.7G 67% /var
/dev/mapper/cl_fog-home xfs 10G 930M 9.1G 10% /home
/dev/mapper/fog_images xfs 1.0T 37G 988G 4% /images
tmpfs tmpfs 379M 0 379M 0% /run/user/27129
tmpfs tmpfs 379M 0 379M 0% /run/user/0
@Tom-Elliott I noticed another update was available yesterday so ran it, rebooted and went to bed. Unfortunately woke up to more Interface not ready messages…
1444 2021-04-26 17:11:15 cd fogproject-dev-branch/
1445 2021-04-26 17:11:18 git pull
1447 2021-04-26 17:11:47 cd bin
1449 2021-04-26 17:12:01 ./installfog.sh
1450 2021-04-26 17:17:26 reboot
...
[04-26-21 5:32:51 pm] Interface not ready, waiting for it to come up:
[04-26-21 5:33:01 pm] Interface not ready, waiting for it to come up:
[04-26-21 5:33:11 pm] Interface not ready, waiting for it to come up:
...
[04-26-21 5:33:21 pm] Interface not ready, waiting for it to come up:
[04-26-21 5:33:31 pm] Interface not ready, waiting for it to come up:
[04-26-21 5:33:41 pm] Interface not ready, waiting for it to come up:
One thing that stands out to me is that the FOG server was rebooted at 5:17 last night, but the first message in the logs was timestamped 5:32. What was happening for those 15 minutes?
I rebooted again this morning and the interface came up immediately. I’ll keep an eye on the logs to see if it goes sideways in a few minutes:
1464 2021-04-27 06:54:30 reboot
...
tail /var/log/fog/fogscheduler.log
[04-27-21 6:54:59 am] Interface Ready with IP Address: [localIP]
[04-27-21 6:54:59 am] Interface Ready with IP Address: 127.0.0.1
[04-27-21 6:54:59 am] Interface Ready with IP Address: 127.0.1.1
[04-27-21 6:54:59 am] Interface Ready with IP Address: fog.my.domain.com
[04-27-21 6:54:59 am] * Starting TaskScheduler Service
[04-27-21 6:54:59 am] * Checking for new items every 60 seconds
[04-27-21 6:54:59 am] * Starting service loop
[04-27-21 6:54:59 am] * 1 task found.
[04-27-21 6:54:59 am] * 1 scheduled task(s) to run.
[04-27-21 6:54:59 am] * 0 power management task(s) to run.
@tom-elliott Good morning, Tom! When I checked logs this morning it was full of ‘waiting for the interface to come up’ messages. I checked the order of my commands on Friday and can confirm that I did bounce the scheduler service after truncating the table:
1430 2021-04-23 15:25:07 mysql -uroot -p
1431 2021-04-23 15:25:27 systemctl restart FOGScheduler
I’m not sure what I missed. I restarted the service this morning and the interface came up. It looks like the previous logs were lost when I restarted the service.
Thank you for checking in!
@tom-elliott said in Daily wake-up routine not working after update:
TRUNCATE TABLE powerManagement
I missed that in your previous post. Thank you! I think this one’s solved!
[04-23-21 3:25:27 pm] * Starting TaskScheduler Service
[04-23-21 3:25:27 pm] * Checking for new items every 60 seconds
[04-23-21 3:25:27 pm] * Starting service loop
[04-23-21 3:25:27 pm] * No tasks found!
[04-23-21 3:26:27 pm] * 1 task found.
[04-23-21 3:26:27 pm] * 1 scheduled task(s) to run.
[04-23-21 3:26:27 pm] * 0 power management task(s) to run.
I’ll find out Monday. Have a great weekend, @Tom-Elliott !
I went ahead and re-created the wakeup cron using the breadcrumbs described and nothing happened in the logs for ~five minutes. So I restarted the service again and now have 79 tasks found:
[04-23-21 3:21:35 pm] * Starting TaskScheduler Service
[04-23-21 3:21:35 pm] * Checking for new items every 60 seconds
[04-23-21 3:21:35 pm] * Starting service loop
[04-23-21 3:21:35 pm] * 79 tasks found.
[04-23-21 3:21:35 pm] * 0 scheduled task(s) to run.
[04-23-21 3:21:35 pm] * 79 power management task(s) to run.
@tom-elliott Group -> Power Management -> Cron time -> WOL. But I deleted it earlier today so I’m not sure why it’s still there.
After restarting the service the logs look like this:
[04-23-21 3:15:06 pm] * Starting TaskScheduler Service
[04-23-21 3:15:06 pm] * Checking for new items every 60 seconds
[04-23-21 3:15:06 pm] * Starting service loop
[04-23-21 3:15:06 pm] * 59 tasks found.
[04-23-21 3:15:06 pm] * 0 scheduled task(s) to run.
[04-23-21 3:15:06 pm] * 59 power management task(s) to run.
@tom-elliott I created the tasks through the GUI to wake the groups up. After the last update, I see a bunch of this instead:
[04-23-21 2:51:51 pm] Interface not ready, waiting for it to come up:
@tom-elliott said in Daily wake-up routine not working after update:
What does it look like now?
[04-23-21 1:27:29 pm] Interface Ready with IP Address: [privateIP]
[04-23-21 1:27:29 pm] Interface Ready with IP Address: 127.0.0.1
[04-23-21 1:27:29 pm] Interface Ready with IP Address: 127.0.1.1
[04-23-21 1:27:29 pm] Interface Ready with IP Address: fog.my.domain.com
[04-23-21 1:27:29 pm] * Starting TaskScheduler Service
[04-23-21 1:27:29 pm] * Checking for new items every 60 seconds
[04-23-21 1:27:29 pm] * Starting service loop
[04-23-21 1:27:29 pm] * 59 tasks found.
[04-23-21 1:27:29 pm] * 0 scheduled task(s) to run.
[04-23-21 1:27:29 pm] * 59 power management task(s) to run.
@tom-elliott Well this is getting interesting:
[04-23-21 11:58:31 am]
==================================
=== ==== ===== ====
=== ========= == === == ===
=== ======== ==== == ==== ===
=== ======== ==== == =========
=== ==== ==== == =========
=== ======== ==== == === ===
=== ======== ==== == ==== ===
=== ========= == === == ===
=== ========== ===== ====
==================================
===== Free Opensource Ghost ======
==================================
============ Credits =============
= https://fogproject.org/Credits =
==================================
== Released under GPL Version 3 ==
==================================
[04-23-21 11:58:31 am] Interface Ready with IP Address: [privateIP]
[04-23-21 11:58:31 am] Interface Ready with IP Address: 127.0.0.1
[04-23-21 11:58:31 am] Interface Ready with IP Address: 127.0.1.1
[04-23-21 11:58:31 am] Interface Ready with IP Address: fog.my.domain.com
[04-23-21 11:58:31 am] * Starting TaskScheduler Service
[04-23-21 11:58:31 am] * Checking for new items every 60 seconds
[04-23-21 11:58:31 am] * Starting service loop
[04-23-21 11:58:31 am] * 59 tasks found.
[04-23-21 11:58:31 am] * Running loop of all tasks.
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 52 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 54 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 62 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 65 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 67 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 81 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 82 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 83 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 84 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 85 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 86 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
[04-23-21 11:58:31 am] * 87 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
...
[04-23-21 11:58:31 am] * 94 Does not appear to be valid
[04-23-21 11:58:31 am] * Attempting to run checks for PowerManagement
It also helps that you guys have made the upgrade process so painless 
@tom-elliott said in Daily wake-up routine not working after update:
I know I’m asking a lot, but please bare with me.
You are so generous with your time and resources, and polite to boot. I can’t tell you how much I appreciate you digging into this particular issue for me.
I’ve been waking up early to click the WakeUp link for all our employees once the functionality stopped working.
Here’s what’s in the logs after updating to the .70 branch:
...
===== Free Opensource Ghost ======
==================================
============ Credits =============
= https://fogproject.org/Credits =
==================================
== Released under GPL Version 3 ==
==================================
[04-23-21 11:37:32 am] Interface Ready with IP Address: [privateIP]
[04-23-21 11:37:32 am] Interface Ready with IP Address: 127.0.0.1
[04-23-21 11:37:32 am] Interface Ready with IP Address: 127.0.1.1
[04-23-21 11:37:32 am] Interface Ready with IP Address: fog.my.domain.com
[04-23-21 11:37:32 am] * Starting TaskScheduler Service
[04-23-21 11:37:32 am] * Checking for new items every 60 seconds
[04-23-21 11:37:32 am] * Starting service loop
[04-23-21 11:37:32 am] * 59 tasks found.
Again, I deleted the scheduled tasks I had in there so I’m not sure where the 59 task count is coming from.
@tom-elliott Good morning Tom! Thanks for your thoughts on this, and for checking into the scheduler’s code. There were two scheduled tasks - one for a group and one for an individual PC. I deleted both of them before seeing your latest update, so now there are zero scheduled tasks:
I ran the upgrade to version 1.5.9.69 and this is the contents of my log file:
/var/log/fog/fogscheduler.log
[04-23-21 7:57:42 am]
==================================
=== ==== ===== ====
=== ========= == === == ===
=== ======== ==== == ==== ===
=== ======== ==== == =========
=== ==== ==== == =========
=== ======== ==== == === ===
=== ======== ==== == ==== ===
=== ========= == === == ===
=== ========== ===== ====
==================================
===== Free Opensource Ghost ======
==================================
============ Credits =============
= https://fogproject.org/Credits =
==================================
== Released under GPL Version 3 ==
==================================
[04-23-21 7:57:42 am] Interface Ready with IP Address: [privateIP]
[04-23-21 7:57:42 am] Interface Ready with IP Address: 127.0.0.1
[04-23-21 7:57:42 am] Interface Ready with IP Address: 127.0.1.1
[04-23-21 7:57:42 am] Interface Ready with IP Address: fog.my.domain.com
[04-23-21 7:57:42 am] * Starting TaskScheduler Service
[04-23-21 7:57:42 am] * Checking for new items every 60 seconds
[04-23-21 7:57:42 am] * Starting service loop
[04-23-21 7:57:42 am] * 59 tasks found.
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
[04-23-21 7:57:42 am] * Task run time: Mon, 26 Apr 2021 06:00:00 -0500
Thank you!
@Tom-Elliott Just an update that this is still occurring after the latest update to dev branch 1.5.9.67. The scheduler log spits out a one-time block of text upon service start, but nothing else:
/var/log/fog/fogscheduler.log
[04-21-21 4:38:37 pm]
==================================
=== ==== ===== ====
=== ========= == === == ===
=== ======== ==== == ==== ===
=== ======== ==== == =========
=== ==== ==== == =========
=== ======== ==== == === ===
=== ======== ==== == ==== ===
=== ========= == === == ===
=== ========== ===== ====
==================================
===== Free Opensource Ghost ======
==================================
============ Credits =============
= https://fogproject.org/Credits =
==================================
== Released under GPL Version 3 ==
==================================
[04-21-21 4:38:37 pm] Interface Ready with IP Address: [localIP]
[04-21-21 4:38:37 pm] Interface Ready with IP Address: 127.0.0.1
[04-21-21 4:38:37 pm] Interface Ready with IP Address: 127.0.1.1
[04-21-21 4:38:37 pm] Interface Ready with IP Address: fog.my.domain.com
[04-21-21 4:38:37 pm] * Starting TaskScheduler Service
[04-21-21 4:38:37 pm] * Checking for new items every 60 seconds
[04-21-21 4:38:37 pm] * Starting service loop
[04-21-21 4:38:37 pm] * 61 tasks found.
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 05:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
[04-21-21 4:38:37 pm] * Task run time: Thu, 22 Apr 2021 06:00:00 -0500
Seems similar to @Petěrko’s issue over at:
https://forums.fogproject.org/topic/15342/scheduled-wake-on-lan
@petěrko I’m having the same problem with the latest dev branches. It was working fine before I updated. See:
https://forums.fogproject.org/topic/15279/daily-wake-up-routine-not-working-after-update/
Any way to merge topics?
After a fresh look these seem like separate issues. Sorry for the interruption.
Yes, that solved it. Thanks for the quick work, @Sebastian-Roth!
How do I mark a topic as Solved? Or is that a moderator-only task?
@Sebastian-Roth as requested:
[Cent-7:root@fog bin]# grep packages /opt/fog/.fogsettings
packages='bc curl gcc gcc-c++ genisoimage gzip httpd lftp m4 make mariadb mariadb-server mod_ssl mtools net-tools nfs-utils php php-bcmath php-cli php-common php-fpm php-gd php-ldap php-mbstring php-mysqlnd php-process syslinux tar tftp-server unzip vsftpd wget xinetd xz-devel'
Also:
[Cent-7:root@fog bin]# yum install mariadb
...
Package 1:mariadb-5.5.68-1.el7.x86_64 is obsoleted by MariaDB-client-10.5.9-1.el7.centos.x86_64 which is already installed
Nothing to do
[Cent-7:root@fog bin]# yum install mariadb-server
...
Package 1:mariadb-server-5.5.68-1.el7.x86_64 is obsoleted by MariaDB-server-10.5.9-1.el7.centos.x86_64 which is already installed
Nothing to do
Thank you!
My MariaDB was on 5.5 on a CentOS 7 fully-updated server and I wanted a newer version, so I removed it:
Packages Altered:
Erase mariadb-server-1:5.5.68-1.el7.x86_64 @base
I installed the MariaDB repository from:
https://downloads.mariadb.com/MariaDB/mariadb_repo_setup
Packages Altered:
Obsoleting MariaDB-client-10.5.9-1.el7.centos.x86_64 @mariadb-main
Dep-Install MariaDB-common-10.5.9-1.el7.centos.x86_64 @mariadb-main
Obsoleting MariaDB-compat-10.5.9-1.el7.centos.x86_64 @mariadb-main
Install MariaDB-server-10.5.9-1.el7.centos.x86_64 @mariadb-main
Dep-Install boost-program-options-1.53.0-28.el7.x86_64 @base
Dep-Install galera-4-26.4.7-1.el7.centos.x86_64 @mariadb-main
Obsoleted mariadb-1:5.5.68-1.el7.x86_64 @base
Obsoleted mariadb-libs-1:5.5.68-1.el7.x86_64 @base
Dep-Install pcre2-10.23-2.el7.x86_64 @base
Dep-Install socat-1.7.3.2-2.el7.x86_64 @base
and verified the database is operational:
# mysql -uroot -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 611444
Server version: 10.5.9-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| fog |
| information_schema |
| mysql |
| performance_schema |
+--------------------+
4 rows in set (0.001 sec)
MariaDB [(none)]>
I did a new get pull for the latest dev branch of fogproject but the installer does not recognize my MariaDB installation:
* Checking package: make......................................OK
* Checking package: mariadb...................................Failed!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! The installer was not able to run all the way to the end as !!
!! something has caused it to fail. The following few lines are !!
!! from the error log file which might help us figure out what's !!
!! wrong. Please add this information when reporting an error. !!
!! As well you might want to take a look at the full error log !!
!! in /files/git/fogproject-dev-branch/bin/error_logs/fog_error_1.5.9.65.log !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
httpd-2.4.6-97.el7.centos.x86_64
lftp-4.4.8-12.el7_8.1.x86_64
m4-1.4.16-10.el7.x86_64
make-3.82-24.el7.x86_64
package mariadb is not installed
Also upgraded the database:
# mysql_upgrade -uroot -p
Enter password:
This installation of MariaDB is already upgraded to 10.5.9-MariaDB, use --force if you still need to run mysql_upgrade
more details:
# mysql -V
mysql Ver 15.1 Distrib 10.5.9-MariaDB, for Linux (x86_64) using readline 5.1
# rpm -qa | grep -i maria
MariaDB-client-10.5.9-1.el7.centos.x86_64
php-williamdes-mariadb-mysql-kbs-1.2.12-1.el7.remi.noarch
MariaDB-common-10.5.9-1.el7.centos.x86_64
MariaDB-server-10.5.9-1.el7.centos.x86_64
MariaDB-compat-10.5.9-1.el7.centos.x86_64
So I guess my question is, how does installfog.sh verify whether or not MariaDB is available?