CPU Usage high

[Sebastian Roth]

@Taspharel This is even after rebooting the FOG server? Still FOGTaskSchedule with high CPU? Do you see anything useful in the apache logs (see my signature on where to find those)? As well, post the contents of your /var/log/fog/fogscheduler.log here.

[Quazz]

@taspharel said in CPU Usage high:

no Tasks scheduled (due to it still not working for us, but thats a different story)

Seems rather relevant to me if FOGTaskSchedule is taking up so much CPU while not working.

What happens when you try to schedule a task?

When you restart apache, how long does it function normally before CPU goes to 100?

[Wayne Workman]

@sebastian-roth said in CPU Usage high:

As well, post the contents of your /var/log/fog/fogscheduler.log here.

That log will be most important in this situation. Have logs moved? I remember them being inside /opt/fog/log ?

[Sebastian Roth]

@Wayne-Workman This is a link… /opt/fog/log is actually the place of the logs. As most people are used to /var/log I mostly use this.

[Taspharel]

Fogscheduler Log:

[12-13-17 11:13:05 am]
==================================
===        ====    =====      ====
===  =========  ==  ===   ==   ===
===  ========  ====  ==  ====  ===
===  ========  ====  ==  =========
===      ====  ====  ==  =========
===  ========  ====  ==  ===   ===
===  ========  ====  ==  ====  ===
===  =========  ==  ===   ==   ===
===  ==========    =====      ====
==================================
===== Free Opensource Ghost ======
==================================
============ Credits =============
= https://fogproject.org/Credits =
==================================
== Released under GPL Version 3 ==
==================================


[12-13-17 11:13:05 am] Interface Ready with IP Address: 127.0.0.1
[12-13-17 11:13:05 am] Interface Ready with IP Address: 127.0.1.1
[12-13-17 11:13:05 am] Interface Ready with IP Address: 192.168.43.17
[12-13-17 11:13:05 am] Interface Ready with IP Address: fogserver
[12-13-17 11:13:05 am]  * Starting TaskScheduler Service
[12-13-17 11:13:05 am]  * Checking for new items every 60 seconds
[12-13-17 11:13:05 am]  * Starting service loop
[12-13-17 11:13:05 am]  * 3 tasks found.


**And the Apache error log:**

[Mon Jan 01 03:38:22.208005 2018] [auth_digest:notice] [pid 12312] AH01757: generating secret for digest authentication ...
[Mon Jan 01 03:38:22.208797 2018] [lbmethod_heartbeat:notice] [pid 12312] AH02282: No slotmem from mod_heartmonitor
[Mon Jan 01 03:38:22.225832 2018] [mpm_prefork:notice] [pid 12312] AH00163: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/5.6.32 configured -- resuming normal operations
[Mon Jan 01 03:38:22.225842 2018] [core:notice] [pid 12312] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Wed Jan 03 11:18:17.743911 2018] [:error] [pid 10603] [client 192.168.43.93:25534] PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 35505 bytes) in /var/www/html/fog/lib/fog/fogcontroller.class.php on line 261, referer: http://fog.lfdw.local/fog/management/index.php?node=host&sub=edit&id=165
[Wed Jan 03 11:18:48.656285 2018] [:error] [pid 23214] [client 192.168.43.93:25548] PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 35505 bytes) in /var/www/html/fog/lib/fog/fogcontroller.class.php on line 261, referer: http://fog.lfdw.local/fog/management/index.php?node=host&sub=edit&id=165
[Wed Jan 03 11:20:26.383096 2018] [:error] [pid 23214] [client 192.168.43.93:25585] PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 35505 bytes) in /var/www/html/fog/lib/fog/fogcontroller.class.php on line 261, referer: http://fog.lfdw.local/fog/management/index.php?node=host&sub=edit&id=165
[Wed Jan 03 11:26:43.136156 2018] [:error] [pid 23402] [client 192.168.43.93:25747] PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 35505 bytes) in /var/www/html/fog/lib/fog/fogcontroller.class.php on line 261, referer: http://fog.lfdw.local/fog/management/index.php?node=host&sub=edit&id=165
[Wed Jan 03 11:29:43.869919 2018] [:error] [pid 10603] [client 192.168.43.93:25822] PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 35505 bytes) in /var/www/html/fog/lib/fog/fogcontroller.class.php on line 261, referer: http://fog.lfdw.local/fog/management/index.php?node=host&sub=edit&id=165
[Wed Jan 03 11:48:19.361457 2018] [:error] [pid 10603] [client 192.168.43.93:26266] PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 35505 bytes) in /var/www/html/fog/lib/fog/fogcontroller.class.php on line 261, referer: http://fog.lfdw.local/fog/management/index.php?node=host&sub=edit&id=165
[Wed Jan 03 11:54:39.077064 2018] [:error] [pid 23270] [client 192.168.43.93:26420] PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 35505 bytes) in /var/www/html/fog/lib/fog/fogcontroller.class.php on line 261, referer: http://fog.lfdw.local/fog/management/index.php?node=host&sub=edit&id=165
[Wed Jan 03 11:56:01.568297 2018] [:error] [pid 23305] [client 192.168.43.93:26450] PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 35505 bytes) in /var/www/html/fog/lib/fog/fogcontroller.class.php on line 261, referer: http://fog.lfdw.local/fog/management/index.php?node=host&sub=edit&id=165
[Wed Jan 03 12:20:25.820408 2018] [:error] [pid 23305] [client 192.168.43.93:26935] PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 16384 bytes) in /var/www/html/fog/lib/fog/fogcontroller.class.php on line 261, referer: http://fog.lfdw.local/fog/management/index.php?node=host&sub=edit&id=165
[Wed Jan 03 12:37:30.868478 2018] [:error] [pid 23305] [client 192.168.43.93:27362] PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 49152 bytes) in /var/www/html/fog/lib/fog/fogcontroller.class.php on line 261, referer: http://fog.lfdw.local/fog/management/index.php?node=host&sub=edit&id=127

Regarding the schedules:
I still have the problem that when trying to show my scheduled tasks the web gui hangs and I cannot open any more pages on the webgui until I delete my browsers data (cookies, etc.) - tried with several different browsers and seems (at least according to what i took from the thread where I posted that) to be a problem with my setup only … since we dont necessarily need the scheduled tasks I just ignored it so far.

Will try a reboot now.

Update:
Rebooting the machine puts the FOGTaskSchedule at the top with 100 % CPU again, 0.2 % Mem … the httpd ones are still at the top but with much lower cpu values, sometimes going down to 0.

[Sebastian Roth]

@Taspharel Please run the following database queries to see what tasks are still active.

shell> mysql -u root -p
Enter password:
...
mysql> use fog;
...
mysql> SELECT * FROM scheduledTasks WHERE stActive=1;
...
mysql> SELECT * FROM powerManagement WHERE pmAction LIKE '%wol%';
...
mysql> quit;
shell>

[Taspharel]

First command results in:

Second one:

[Sebastian Roth]

@Taspharel As you said you don’t need those tasks I’d either delete those from the database or simply unset the active bit:

either disable:

shell> mysql -u root -p
Enter password:
...
mysql> use fog;
...
mysql> UPDATE scheduledTasks SET stActive=0 WHERE stActive=1;
...
mysql> quit;
shell>

or delete:

shell> mysql -u root -p
Enter password:
...
mysql> use fog;
...
mysql> DELETE FROM scheduledTasks WHERE stActive=1;
...
mysql> quit;
shell>

Around 60 seconds later the CPU load should drop. If not you can try restarting the server.

[Taspharel]

Thanks for the help, disabled the active bit and after a reboot ‘top’ no longer shows any process with high CPU usage in idle state.

In the long term the question remains: Why did that happen at all and is it related to our error regarding the web-gui “freeze” when looking at scheduled tasks?

Update:
Just checked: I can now open the “Scheduled Tasks” website in the FOG-Gui without problems …

[Sebastian Roth]

@taspharel said in CPU Usage high:

In the long term the question remains: Why did that happen at all and is it related to our error regarding the web-gui “freeze” when looking at scheduled tasks?

Well it might be related as in the “freeze” might have been caused by the same “zombie” jobs. But I don’t really know.

Looking at the three tasks in the database again I see that all of them had stImageID zero. Don’t think that is valid. Not sure if that was causing it.