Daily wake-up routine not working after update

FOG Problems
Log in to reply
  • A

    Hi Team,

    I updated to the latest dev branch earlier this week and now our scheduled WoL isn’t working. I can still trigger it manually so I know the WoL packets are not the issue.
    I tried deleting the task and creating a new one but no change.

    Just looked at the /var/log/fog/fogscheduler.log and the latest entry is from the date I updated the system:

    [03-22-21 10:30:55 am] * Task run time: Tue, 23 Mar 2021 06:00:00 -0500

    Looks like a service isn’t running, but checking the scheduler it appears fine:

    [Cent-7:root@fog ~]# systemctl status FOGScheduler.service
● FOGScheduler.service - FOGScheduler
   Loaded: loaded (/usr/lib/systemd/system/FOGScheduler.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2021-03-19 14:31:56 CDT; 5 days ago
 Main PID: 2955 (php)
   Memory: 16.6M
   CGroup: /system.slice/FOGScheduler.service
           ├─2955 php /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler
           └─3124 php /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler

    Using my favorite command I restarted the FOG services:

    for i in FOGSnapinHash.service FOGScheduler.service  FOGImageSize.service FOGSnapinReplicator.service FOGMulticastManager.service FOGImageReplicator.service FOGPingHosts.service;do systemctl restart $i; done

    Now I have this at the beginning of the log:

    [03-25-21 9:06:22 am] * Starting TaskScheduler Service
[03-25-21 9:06:22 am] * Checking for new items every 60 seconds
[03-25-21 9:06:22 am] * Starting service loop
[03-25-21 9:06:22 am] * 61 tasks found.

    Do I need to worry about the 61 tasks found? Anything else I need to check to keep the service active?

    Thank you!

    0 A 1 Reply
  • A

    @Tom-Elliott any other places to check? I don’t mind clearing out all scheduled tasks since I don’t have much in there. Just not sure how to do a safe reset.

    0
  • A

    @tom-elliott

    MariaDB [fog]> UPDATE tasks SET taskStateID=5 WHERE taskStateID IN ('0','',NULL,'1','2','3') AND taskTypeID=14;
Query OK, 0 rows affected, 1 warning (0.000 sec)
Rows matched: 0  Changed: 0  Warnings: 1

    And the logs after restarting the service:

    /var/log/fog/fogscheduler.log
[03-31-21 7:04:29 am] 
==================================
=== ==== ===== ====
=== ========= == === == ===
=== ======== ==== == ==== ===
=== ======== ==== == =========
=== ==== ==== == =========
=== ======== ==== == === ===
=== ======== ==== == ==== ===
=== ========= == === == ===
=== ========== ===== ====
==================================
===== Free Opensource Ghost ======
==================================
============ Credits =============
= https://fogproject.org/Credits =
==================================
== Released under GPL Version 3 ==
==================================


[03-31-21 7:04:29 am] Interface Ready with IP Address: [localIP]
[03-31-21 7:04:29 am] Interface Ready with IP Address: 127.0.0.1
[03-31-21 7:04:29 am] Interface Ready with IP Address: 127.0.1.1
[03-31-21 7:04:29 am] Interface Ready with IP Address: fog.domain.com
[03-31-21 7:04:29 am] * Starting TaskScheduler Service
[03-31-21 7:04:29 am] * Checking for new items every 60 seconds
[03-31-21 7:04:29 am] * Starting service loop
[03-31-21 7:04:29 am] * 61 tasks found.
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 05:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
[03-31-21 7:04:29 am] * Task run time: Thu, 01 Apr 2021 06:00:00 -0500
    0 A 1 Reply
  • Tom Elliott
    Senior Developer

    @altitudehack If you can try:

    UPDATE tasks SET taskStateID=5 WHERE taskStateID IN ('0','',NULL,'1','2','3') AND taskTypeID=14;

    Then restart FOGScheduler

    If you can let us know what your log shows?

    Thank you,

    0 A 1 Reply
  • Tom Elliott
    Senior Developer

    @altitudehack All of the tasks are purely WOL tasks, correct?

    Just want clarification as I want to try to clear out the DB of erroneous entries that I suspect are playing a bit of a role in the issue.

    0
  • A

    @Tom-Elliott I rebooted the server this morning about six minutes prior to the scheduled wake-up time. Still nothing showing in the schedule log:

    /var/log/fog/fogscheduler.log
[03-31-21 5:54:00 am] 
==================================
=== ==== ===== ====
=== ========= == === == ===
=== ======== ==== == ==== ===
=== ======== ==== == =========
=== ==== ==== == =========
=== ======== ==== == === ===
=== ======== ==== == ==== ===
=== ========= == === == ===
=== ========== ===== ====
==================================
===== Free Opensource Ghost ======
==================================
============ Credits =============
= https://fogproject.org/Credits =
==================================
== Released under GPL Version 3 ==
==================================


[03-31-21 5:54:00 am] Interface Ready with IP Address: [privateIP]
[03-31-21 5:54:00 am] Interface Ready with IP Address: 127.0.0.1
[03-31-21 5:54:00 am] Interface Ready with IP Address: 127.0.1.1
[03-31-21 5:54:00 am] Interface Ready with IP Address: fog.domain.com
[03-31-21 5:54:00 am] * Starting TaskScheduler Service
[03-31-21 5:54:00 am] * Checking for new items every 60 seconds
[03-31-21 5:54:00 am] * Starting service loop
[03-31-21 5:54:00 am] * 61 tasks found.
[03-31-21 5:54:00 am] * Task run time: Thu, 01 Apr 2021 05:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500
[03-31-21 5:54:00 am] * Task run time: Wed, 31 Mar 2021 06:00:00 -0500

    I still have the entries showing up in scheduled tasks:
    scheduled.png

    0 Tom Elliott 2 Replies
  • A

    @tom-elliott

    # systemctl status -l FOGScheduler.service
● FOGScheduler.service - FOGScheduler
   Loaded: loaded (/usr/lib/systemd/system/FOGScheduler.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2021-03-30 09:24:36 CDT; 2h 20min ago
 Main PID: 14299 (php)
   Memory: 13.6M
   CGroup: /system.slice/FOGScheduler.service
           ├─14299 php /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler
           └─14300 php /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler

Mar 30 09:24:36 fog.my.domain.com systemd[1]: Started FOGScheduler.
    0
  • Tom Elliott
    Senior Developer

    @altitudehack What about output of:

    systemctl status -l FOGScheduler.service?

    0 A 1 Reply
  • A

    @tom-elliott said in Daily wake-up routine not working after update:

    log/php-fpm/www-error.log

    There was no output from this or any logfile in that directory during a restart of the service, which happened at 09:24:36:

    # ls -lht /var/log/php-fpm/
total 276K
-rwxr-xr-x. 1 root   root    15K Mar 30 07:50 error.log
-rwxr-xr-x. 1 root   root    59K Mar 28 06:59 error.log-20210328
-rwxr-xr-x. 1 root   root    58K Mar 21 03:27 error.log-20210321
-rwxr-xr-x. 1 root   root    60K Mar 14 07:53 error.log-20210314
-rwxr-xr-x. 1 root   root    60K Mar  7 06:20 error.log-20210307
-rwxr-xr-x. 1 apache apache    0 Mar  4 05:13 www-error.log
-rwxr-xr-x. 1 apache apache 1.6K Mar  3 08:03 www-error.log-20210304
-rwxr-xr-x. 1 apache apache 5.6K Feb 19 11:25 www-error.log-20210221
-rwxr-xr-x. 1 apache apache 1.6K Feb 16 08:17 www-error.log-20210217
-rwxr-xr-x. 1 apache apache 1.6K Jan 31 17:07 www-error.log-20210201
    0 Tom Elliott 1 Reply
  • Tom Elliott
    Senior Developer

    @altitudehack I’m not quite sure whats going on but it feels like a php-fpm error.

    Can you restart FOGScheduler (systemctl restart FOGScheduler) while tailing the output of:
    /var/log/php-fpm/www-error.log ( at least that’s where it is on Centos, unsure for debian )

    0 A 1 Reply
  • A

    Nothing has processed since the service restart on the 25th. This is the entirety of the scheduler log as of March 30th:

    /var/log/fog/fogscheduler.log
[03-25-21 2:39:17 pm] 
==================================
=== ==== ===== ====
=== ========= == === == ===
=== ======== ==== == ==== ===
=== ======== ==== == =========
=== ==== ==== == =========
=== ======== ==== == === ===
=== ======== ==== == ==== ===
=== ========= == === == ===
=== ========== ===== ====
==================================
===== Free Opensource Ghost ======
==================================
============ Credits =============
= https://fogproject.org/Credits =
==================================
== Released under GPL Version 3 ==
==================================


[03-25-21 2:39:17 pm] Interface Ready with IP Address: [privateIP]
[03-25-21 2:39:17 pm] Interface Ready with IP Address: 127.0.0.1
[03-25-21 2:39:17 pm] Interface Ready with IP Address: 127.0.1.1
[03-25-21 2:39:17 pm] Interface Ready with IP Address: [fqdn]
[03-25-21 2:39:17 pm] * Starting TaskScheduler Service
[03-25-21 2:39:17 pm] * Checking for new items every 60 seconds
[03-25-21 2:39:17 pm] * Starting service loop
[03-25-21 2:39:17 pm] * 61 tasks found.
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 05:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500
[03-25-21 2:39:17 pm] * Task run time: Fri, 26 Mar 2021 06:00:00 -0500

    Is there anywhere else I can look?
    Thanks!

    0 Tom Elliott 1 Reply

349
Online

8.1k
Users

15.0k
Topics

141.4k
Posts