1.3.4 - high cpu load - client login



  • Server
    • FOG Version: 1.3.4 svn 6064
    • OS: Ubuntu 14.04
    Client
    • Service Version: 11.x
    • OS: Win 7
    Description

    Seeing high cpu utilization on the fog server. The server gets overwhelmed to the point of not being able to respond timely to pxe requests.

    AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

    The log is full of these:

    PHP Fatal error: Uncaught Error: Call to a member function isValid() on null in /var/www/html/fog/lib/client/registerclient.class.php:67\nStack trace:\n#0 /var/www/html/fog/lib/fog/fogpage.class.php(2701): RegisterClient->json()\n#1 /var/www/html/fog/lib/fog/fogpage.class.php(262): FOGPage->requestClientInfo()\n#2 /var/www/html/fog/lib/fog/processlogin.class.php(57): FOGPage->__construct(’’)\n#3 [internal function]: ProcessLogin->__construct()\n#4 /var/www/html/fog/lib/fog/fogbase.class.php(406): ReflectionClass->newInstanceArgs(Array)\n#5 /var/www/html/fog/management/index.php(29): FOGBase::getClass(‘ProcessLogin’)\n#6 {main}\n thrown in /var/www/html/fog/lib/client/registerclient.class.php on line 67



  • @Wayne-Workman
    Hi,
    Ok i understand.
    Actually majority off our classroom reboot at the same time for software update in night (same time is residual effect with green fog on old version).
    I will adjust power management (good amelioration in this version) for spread out this reboot.

    Thanks.



  • 1.3.5-RC 10 and working fine.

    All clients are back on, I’ve already reduced the client check in time, and the load was fine for the start of the school day.

    The Image replicator service is still off.


  • Moderator

    @Florent said in 1.3.4 - high cpu load - client login:

    How it’s possible to modify the random staggering range for spread over 10 minutes for example ?

    The staggering is not meant to be adjustable, what’s meant to be adjustable is the FOG_CLIENT_CHECKIN_TIME, this governs how often the FOG Client should check with the server. Assuming 10,000 computers power on all at exactly the same moment, over the course of the day the random staggering will cause all clients to spread out evenly with their check-ins. But nobody has 10,000 systems all powering on at the same second. Generally, organizations either A. leave their computers on all the time or B. allow the end-users to power them on.

    Also asking @Joe-Schmitt if he could give further explanation.



  • @ablohowiak @Tom-Elliott

    Hi,

    Today it’s better but all the 30 minutes we have pic of 2 minutes.

    In wiki it’s write (https://wiki.fogproject.org/wiki/index.php?title=FOG_Client):

    The frequency of the checkin-time determines how quickly the FOG Client will receive instructions from the FOG Server. If an image deployment is scheduled for a computer that is turned on, with a checkin-time of 60 seconds, means the FOG Client may begin initiating the task anywhere from 0 to 60 seconds + the random staggering time that is added. This same concept would apply to immediate power management tasks, snapin tasks, capture tasks, and so on. Scheduled tasks are not affected by this behavior, and if the target system is on when the scheduled task is to be ran, this will happen on time

    How it’s possible to modify the random staggering range for spread over 10 minutes for example ?

    Thanks.



  • @ablohowiak
    Oh very thanks it’s works !
    I was thinking test this before but maybe not with “imagereplicatorfogenabled” at off.
    After few minutes cpu is ok.

    Thanks.



  • Yes.
    0_1488311359134_upload-489b07bb-a854-4310-a7a9-8990ae16e77d



  • @ablohowiak said in 1.3.4 - high cpu load - client login:

    So no improvement with imagereplicatorglobalenabled off?

    No :(

    With this issue I’ve increase my client communication up to 30 minutes. >I will reduce when I know things have stabilized.

    Where you have up to 30 minutes your client communication , “checkin time parameter” ?



  • So no improvement with imagereplicatorglobalenabled off?

    With this issue I’ve increase my client communication up to 30 minutes. I will reduce when I know things have stabilized.

    In Ubuntu, see which mpm you have enabled. There should only be one in /etc/apache2/mods-enabled. That’s the file you would edit to increase the MaxRequestWorkers. Make a copy of the file before editing.



  • @UWPVIOLATOR said in 1.3.4 - high cpu load - client login:

    Increased MaxRequestWorkers from 150 to 500 in mpm_prefork_module
    Hi,
    Where you have put this parameter because i try in my apache2.conf in prefork module but not valid after i restart apache2



  • @Tom-Elliott Yes imagereplicatorglobalenabled is not checked



  • Tom,
    We added back about 75% of our clients and the load has remained stable and UI responsive. I was trying to update to 1.3.5-RC10, but the install failed.

    • Downloading inits, kernels, and the fog client…Failed!

    Feb 28 12:47:24 FogDB systemd[1]: Starting MySQL Community Server…
    Feb 28 12:47:26 FogDB systemd[1]: Started MySQL Community Server.
    mysql: [Warning] Using a password on the command line interface can be insecure.
    ERROR 1045 (28000): Access denied for user ‘root’@‘localhost’ (using password: YES)
    mysql: [Warning] Using a password on the command line interface can be insecure.
    ERROR 1045 (28000): Access denied for user ‘root’@‘localhost’ (using password: YES)

    At this point there’s basically no fog site in apache. I’m reverting back to my last snapshot.


  • Senior Developer

    @Florent In the GUI.

    FOG Configuration Page->FOG Settings->FOG Linux Service Enabled



  • @UWPVIOLATOR
    Just do this on Web interface / fog settings or after restart apache ?



  • @Florent

    Do you see something like this in your processes on your FOG server?

    We Disabled IMAGEREPLICATORGLOBALENABLED until Tom fixes the image checking in the next RC.



  • @Tom-Elliott
    Thanks for your response (my english is not very good).

    Yes we have high CPU usage since we have deploy the new client (0.11.9) with GPO.

    We have try to modify FOG_CLIENT_CHECKIN_TIME but we think value over 60 seconds are no effects.
    In our client log we see in general a contact server every 60-200 seconds.
    We have more than 1500 clients.

    If the problem is here is it possible to modify checkin time to 15 minutes ?

    Or if the problem is not this where i can find informations for identify in detail the source of the problem ?

    Regards


  • Senior Developer

    @Florent What is:

    “We have the same problem.”?

    I ask because there seems to be multiple issues being described in this thread, while the primary issue was related to High CPU. Are you referring to High CPU usage being an issue?



  • @Tom-Elliott said in 1.3.4 - high cpu load - client login:

    11

    Hi,
    We have the same problem.
    Is this problem is resolved in RC10 ?
    Or when RC11 available ?

    Regards.



  • @Tom-Elliott

    One thing we notice that is still not working is WOL to Groups. WOL works for individual hosts but not for WOL to a Group. Also Report Management does not return anything for any of the reports.

    0_1487947390955_upload-5fe577c4-0a56-45fb-8c67-d85dafcc384a



  • We had <25% of clients enabled this morning. We need to clean up the snapshots on the server before we can update. It might be this afternoon or Monday.

    We’re still going to re-enable clients in steps just to be safe. By Tuesday morning all clients should be running again.

    Thanks again for all your time and effort!


Log in to reply
 

381
Online

39.3k
Users

11.0k
Topics

104.4k
Posts

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.