CPU Usage 100%

[anthony.delarosa]

this is the error in the fog_error_log

Archive: binaries1.4.3.zip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of binaries1.4.3.zip or
binaries1.4.3.zip.zip, and cannot find binaries1.4.3.zip.ZIP, period.

[THEMCV]

@anthony-delarosa Are you running the installer as root/sudo?

[anthony.delarosa]

@themcv i’ve done it through sudo, I have upgrade FOG before and I don’t recall ever doing it as root : S let me give that a shot

[anthony.delarosa]

@themcv so both root and sudo fail at the same part : S

[anthony.delarosa]

one thing I did notice is that https://svn.code.sf.net/p/freeghost/code/tags/1.4.4 does not get binaries1.4.4.zip but instead binaries 1.4.3.zip which is what is looking for so that’s fair but one thing that is different about that zip file is that the owner if root and not fog like the rest of the folders

[THEMCV]

Ooooohh so you’re using svn not git? That might be the problem.

What does git branch bring up if /root/fogproject/ ?

[anthony.delarosa]

I initially set it up with subversion back in the day so i kinda stuck to svn on every upgrade! i’ll give GIT a try

[THEMCV]

Sure. If you want to upgrade to the dev-branch (I would) we can go from there and see if it’s any better.

[anthony.delarosa]

@themcv hey, i upgraded using GIT and now i’m on 1.4.4 and now running top:

and this is vSphere performance for CPU

does FOG generally use that muchCPU MHz? it is the VM that uses most CPU in this VM Host:

Seems like quite a lot to me

[george1421]

@anthony-delarosa How many client computers (with fog client installed) are on your network? With mysql at 27% it sounds like you have quite a few checking in.

[anthony.delarosa]

@george1421 we do, easily over 800 clients! i was not aware that the clients would be checking in with FOG! I thought it was FOG that would contact the host if it needed to communicate something to it

[george1421]

@anthony-delarosa No the fog client works the other way around, the clients check into the master fog server for things to do. They also post inventory info and such.

Since you have a great performance data so far, I wonder if you would be willing to help the fog project by doing something I had on the back burner for a while.

The idea is simply to install php-fpm and have apache use that instead of its internal php engine. I’m interested to know how that impact:

1. Performance of the web gui
2. Impact on performance of the entire fog server CPU usage.

I see you are using an older OS, but the php-fpm package should still be available in epel or in the centos repository. If you have a little time to test this idea let me know and I will put together a tutorial on how to set this up. Reverting back to the default apache php can be done by just commenting out one line. I would do it on my production fog server, but I only have a small campus. I’ve been looking for someone who had around 1000 clients to see if the test was dramatic.

[george1421]

@anthony-delarosa In your case you can in crease the check-in delay to help with cpu usage on your fog server. You can do this with the understanding that it will also take longer for clients to see snapin deployments and other assigned tasks. It will not impact image deployment only fog client tasks. (but I’d like to see the results of the php-fpm test before you make this change if you agree).

[Wayne Workman]

@anthony-delarosa said in CPU Usage 100%:

@george1421 we do, easily over 800 clients! i was not aware that the clients would be checking in with FOG! I thought it was FOG that would contact the host if it needed to communicate something to it

The fog client uses polling to communicate with the fog server, this is detailed here: https://wiki.fogproject.org/wiki/index.php?title=FOG_Client#Polling_Behavior

[anthony.delarosa]

@george1421 hey George! yeah i think i can help you out there! send me the detailed steps i need to take and i can always take a snapshot of this so if all fails i can revert back!

[anthony.delarosa]

@george1421 is this the setting i can increase? the FOG_CHECKIN_TIMEOUT ?

[george1421]

@anthony-delarosa Excellent, thank you.

Here are my thoughts on how to set this up. I’m only posting below for my reference. I want to spin up a Cento 6.7 FOG server tomorrow and test the process out before you implement it. I also want to see if memcache will help with the process too. But I need to do a bit more research first.

# yum install php-fpm
# chkconfig --levels 235 php-fpm on

Edit /etc/php-fpm.d/www.conf to use sockets

# vi /etc/php-fpm.d/www.conf

;listen = 127.0.0.1:9000
listen = /tmp/php5-fpm.sock

listen.owner = apache
listen.group = apache

# service php-fpm start

Setting up Apache and mod_fastcgi

Install the RPMForge repo:

# wget http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
# rpm -ivh rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm

Add some priorities which repo to use:

# yum install yum-priorities
 
# vi /etc/yum.repos.d/epel.repo 
... add the line priority=10 to the [epel] section

# yum install mod_fastcgi

Configure mod_fastcgi

If you have php enabled disable it

# mv /etc/httpd/conf.d/{php.conf,php.conf.disable}

# mkdir /usr/lib/cgi-bin/

# vi /etc/httpd/conf.d/mod_fastcgi.conf

LoadModule fastcgi_module modules/mod_fastcgi.so
 
<IfModule mod_fastcgi.c>
	DirectoryIndex index.php index.html index.shtml index.cgi
	AddHandler php5-fcgi .php
	Action php5-fcgi /php5-fcgi
	Alias /php5-fcgi /usr/lib/cgi-bin/php5-fcgi
	FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi -socket /tmp/php5-fpm.sock -pass-header Authorization
 
	# For monitoring status with e.g. Munin
	<LocationMatch "/(ping|status)">
		SetHandler php5-fcgi-virt
		Action php5-fcgi-virt /php5-fcgi virtual
	</LocationMatch>
</IfModule>

# service httpd restart

[george1421]

@anthony-delarosa Yes that is the right setting, default check in time is 5 minutes. But if you can hold off a day or so until we can test the php-fpm code. I want to see if it makes a difference on the default settings for FOG. In ‘theory’ it should make a dramatic difference in overall FOG performance.

[george1421]

If you have time to help us test this new configuration, please let me know and I will provide instructions.

[anthony.delarosa]

hey @george1421 how ya doing? yeah sure I can help you guys out! can we schedule this for next week? had a backup meltdown n FOG hadn’t been backed up in a few days so Veeam is doing it’s FULL BACKUP and since our FOG server has over 2 TBs of images it’s backing that up now but should be done soon. Next week works?