Can php-fpm make fog web-gui fast

george1421 · Sep 5, 2017, 5:18 PM

Note this is more of a document blog than a question. Its intended to be as basis of the configuration required to enable php-fpm to speed up php code processing.
Note: This post is only about looking into options to speed up the fog web-gui front end and will have NO IMPACT on FOG imaging. I have another thread on what is required to make FOG Imaging faster.

For this post I’m focusing on Centos 7, but I suspect the configuration will be similar for Debian variants.

By default the fog installer installs php-fpm, but the fog installer doesn’t activate apache to use it. The follow steps are what I did to enable php-fpm to see if it would speed up the FOG web-gui. This speed up is not so much for interacting with the web-gui, but for the backend fog client that will check into the web master server every XX minutes to look for new tasks. If you have 1000 computers checking in every 5 minutes (fog default) then you will have (if averaged out) every second you will have 1.3 check ins. But we know that this check in time will be randomized where you might have 20 check ins within one second and nothing for 3 seconds then a flood again for the next.

Edit the mpm configuration file to tell the apache mpm module to use events and not pre-fork
vi /etc/httpd/conf.modules.d/00-mpm.conf
Comment out
#LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
Uncomment
LoadModule mpm_event_module modules/mod_mpm_event.so
Save and exit from 00-mpm.conf
Change to the apache configuration directory
/etc/httpd/conf.d
Copy the standard php.conf file to create a new php-fpm.conf file
cp /etc/httpd/conf.d/php.conf /etc/httpd/conf.d/php-fpm.conf
Rename the original php.conf file so that apache won’t see it during startup
mv /etc/httpd/conf.d/php.conf /etc/httpd/conf.d/php.conf.disabled
Edit the copied php-fpm.conf file
vi /etc/httpd/conf.d/php-fpm.conf
Comment out the following line
#SetHandler application/x-httpd-php
Insert the following line just below the commented out line
SetHandler "proxy:fcgi://127.0.0.1:9000"
Save and exit the php-fpm.conf file
Change to the php-fpm directory
cd /etc/php-fpm.d
Remove the default www.conf in the php-fpm.d directory (this file is created by the fog installer and not used in this setup)
rm /etc/php-fpm.d/www.conf
Create a new file called fog.conf
vi /etc/php-fpm.d/fog.conf
Paste in the following

[fog]
user = apache
group = apache

listen = 127.0.0.1:9000

;listen.owner = apache
;listen.group = apache
;listen.mode = 0660
;listen.acl_users = apache
;listen.acl_groups =

listen.allowed_clients = 127.0.0.1

pm = dynamic
pm.max_children = 50
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 35
;pm.process_idle_timeout = 10s;
pm.max_requests = 500
;pm.status_path = /status
;ping.path = /ping
;ping.response = pong
 
access.log = /var/log/php-fpm/$pool.access.log
;access.format = "%R - %u %t \"%m %r%Q%q\" %s %f %{mili}d %{kilo}M %C%%"
slowlog = /var/log/php-fpm/$pool-slow.log

;php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com
;php_flag[display_errors] = off
php_admin_value[error_log] = /var/log/php-fpm/fog-error.log
php_admin_flag[log_errors] = on
;php_admin_value[memory_limit] = 128M
php_value[session.save_handler] = files
php_value[session.save_path]    = /var/lib/php/session

; we will use these settings when memcache (d) is configured.
;php_value[session.save_handler] = memcached
;php_value[session.save_path] = "127.0.0.1:11211"

php_value[soap.wsdl_cache_dir]  = /var/lib/php/wsdlcache

Save and exit fog.conf
Create a php information page in the web root
vi /var/www/html/info.php
Paste in the following

<html>
<body>

<?php
 phpinfo();
?>

</body>
</html>

Save and exit info.php page.
Change the owner of that page to apache
chown apache:apache /var/www/html/info.php
From a web browser call the info page with http://<fog_server_ip>/info.php. The page should look similar to below
<insert_info_page_image>
Now lets restart both apache and php-fpm
systemctl restart php-fpm
systemctl restart httpd
Give it a few seconds for both services to initialize.
Now call the same info.php page. The page should look similar to below. NOTE: that the Server API variable is now ‘FPM/FastCGI’
<insert_info_page_image>
Now access the FOG management page console
Access a few of the managment pages, you will note that after a few page clicks the pages will respond faster to your page selections.
To confirm that php-fpm is working you can inspect the logs files being created in /var/log/php-fpm directory. The php errors will now be listed there instead of the default apache error log (because php-fpm is now handling the php code).
You may remove the info.php page in the apache root directory. Its no longer needed.
Done (for now, I’m currently looking into the memcache option. But more on that later).

With these updates we have now handed off php code execution to a dedicated php engine and we are no longer relying on apache to execute both web pages management and php code execution. Will this help the sites that have hundreds of clients? I hope so.

Next steps installing memcache

Wayne Workman · Aug 26, 2017, 6:39 PM

@george1421 and I were talking about this, here’s every commit that references phpfpm or php-fpm or php fpm all case-insensitive.

https://github.com/FOGProject/fogproject/commit/72e2ece88b670490db77dcb98837bbadf1d7e862

https://github.com/FOGProject/fogproject/commit/45a468ffae9a3aabd2e145b957c02f00eb4d9f92

https://github.com/FOGProject/fogproject/commit/60c780219648f5c2e7cf0b90768ea68525e6d93b

https://github.com/FOGProject/fogproject/commit/94e81e7e51cf8b56ca78376ce00a29476e9ff5c6

https://github.com/FOGProject/fogproject/commit/889a6d3a10c1ad947932ae0cf1d41e4594391b69

https://github.com/FOGProject/fogproject/commit/b4b2588bac4c7805076fc0e4d8d66ea161f21e34

https://github.com/FOGProject/fogproject/commit/0bd91a5aaf00fb57b450ca207f9500f4c1a00abb

https://github.com/FOGProject/fogproject/commit/59b28d25344a0a7bd9f19aafb183237e30ec858d

https://github.com/FOGProject/fogproject/commit/24f004d9d5a63df46400803728659f1e725f570b

https://github.com/FOGProject/fogproject/commit/e0cf538ae462d8be97d7565c6891640d5b54e712

george1421 · Aug 27, 2017, 8:58 AM

This is a continuation of this thread. This time I added memcache to store apache/php session data in ram instead of on the filesystem.

Install memcache from the remi (repo added during fog installation)
yum install memcached
Edit the memcache config file
vi /etc/sysconfig/memcached
Add in -l 127.0.0.1 into the options parameter. We only want memcache to be available to the local system.

PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="64"
OPTIONS="-l 127.0.0.1"

Enable the memcached service to auto start
systemctl enable memcached
Start the memcached service
systemctl start memcached
Check to ensure memcached service is running in memory
ps aux|grep memcached
Check to see if memcached service is bound to the loopback adapter
netstat -an|grep 11211
Install memcache for php [note: many instructions on the internet say to compile php-memcache using pecl. I found that remi already has a package so compiling it was not necessary. I can’t say this will be the case for all distros]
yum install php-pecl-memcache
Update the fog php-fpm config file to tell php-fpm to use memcache to store session data.
vi /etc/php-fpm.d/fog.conf
Find these lines at the end of the config file and comment them out

php_value[session.save_handler] = files
php_value[session.save_path]    = /var/lib/php/session

Uncomment these lines

;php_value[session.save_handler] = memcached
;php_value[session.save_path]    = "127.0.0.1:11211"

Save and exit the php-fpm config file
Stop and restart apache and php-fpm

systemctl stop httpd
systemctl stop php-fpm
# wait a few seconds for the dust to settle
systemctl start php-fpm
systemctl start httpd

Done

george1421 · Aug 29, 2017, 1:12 AM

Ubuntu 16.04 configuration.

WARNING: These instructions to not work as of now. Everything is configured correctly but I can't seem to login to the FOG site. There has to be something I'm missing. This configuration is close, very close since the login page appears.

Change to the root user
sudo su -
Install the apache fast cgi module
apt-get -y install libapache2-mod-fastcgi
Enable the apache fast cgi module
a2enmod proxy_fcgi
Edit the php configuration file.
vi /etc/apache2/mods-available/php7.1.conf
Comment out the following line
#SetHandler application/x-httpd-php
Insert the following line just below the commented out line
SetHandler "proxy:fcgi://127.0.0.1:9000"
Save and exit the php-fpm.conf file
Change to the php-fpm pool directory
cd /etc/php/7.1/fpm/pool.d
Remove the default www.conf in the pool.d directory (this file is created by the fog installer and not used in this setup)
rm /etc/php/7.1/fpm/pool.d/www.conf
Create a new file called fog.conf
vi /etc/php/7.1/fpm/pool.d/fog.conf
Paste in the following

[fog]
user = www-data
group = www-data

listen = 127.0.0.1:9000

;listen.owner = www-data
;listen.group = www-data
;listen.mode = 0660
;listen.acl_users = apache
;listen.acl_groups =

listen.allowed_clients = 127.0.0.1

pm = dynamic
pm.max_children = 50
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 35
;pm.process_idle_timeout = 10s;
pm.max_requests = 500
;pm.status_path = /status
;ping.path = /ping
;ping.response = pong
 
access.log = /var/log/php-fpm/$pool.access.log
;access.format = "%R - %u %t \"%m %r%Q%q\" %s %f %{mili}d %{kilo}M %C%%"
slowlog = /var/log/php-fpm/$pool-slow.log

;php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com
;php_flag[display_errors] = off
php_admin_value[error_log] = /var/log/php-fpm/fog-error.log
php_admin_flag[log_errors] = on
;php_admin_value[memory_limit] = 128M
php_value[session.save_handler] = files
php_value[session.save_path]    = /var/lib/php/sessions

; we will use these settings when memcache (d) is configured.
;php_value[session.save_handler] = memcached
;php_value[session.save_path] = "127.0.0.1:11211"

php_value[soap.wsdl_cache_dir]  = /var/lib/php/wsdlcache

Save and exit fog.conf
Create the php-fpm log directory
mkdir /var/log/php-fpm
Create a php information page in the fog web root
vi /var/www/html/fog/info.php
Paste in the following

<html>
<body>

<?php
 phpinfo();
?>

</body>
</html>

Save and exit info.php page.
Change the owner of that page to apache
chown www-data:www-data /var/www/html/fog/info.php
From a web browser call the info page with http://<fog_server_ip>/fog/info.php. The page should look similar to below
<insert_info_page_image>
Now lets restart both apache2 and php-fpm
systemctl restart php7.1-fpm apache2
Give it a few seconds for both services to initialize.
Now call the same info.php page. The page should look similar to below. NOTE: that the Server API variable is now ‘FPM/FastCGI’
<insert_info_page_image>
Now access the FOG management page console
Access a few of the managment pages, you will note that after a few page clicks the pages will respond faster to your page selections.
To confirm that php-fpm is working you can inspect the logs files being created in /var/log/php-fpm directory. The php errors will now be listed there instead of the default apache error log (because php-fpm is now handling the php code).
You may remove the info.php page in the fog web root directory. Its no longer needed.
Done

jburleson · Sep 27, 2017, 3:44 PM

@george1421

I think I found the issue here. The php session save path on Ubuntu should be /var/lib/php/sessions. Update the php_value[session.save_path] in /etc/php/7.1/fpm/pool.d/fog.conf and you should be good to go.

george1421 · Sep 27, 2017, 4:17 PM

@jburleson Great news!! I got pulled off that project for some higher priority ones. If you have it working on ubuntu then wonderful. I know its been working solid on my centos system. No negative things as of now and much faster web ui.

I don’t have enough systems on my campus to really stress test the fog server to see if it helps with many clients and fog server loading.

Actually the sessions path is only used until we turn on memcach. But since I couldn’t login I didn’t go to the next step. really cool if that addresses the issue. Thank you

AndrewG78 · Aug 10, 2018, 4:22 PM

HI,
I recently upgraded FOG to 1.5.4 and php to 7.2.8
When I try memcache with
session.save_handler] = memcached
I got this in logs
"
[10-Aug-2018 16:15:49 UTC] PHP Warning: session_destroy(): Trying to destroy uninitialized session in /var/www/html/fog/lib/fog/user.class.php on line 503
[10-Aug-2018 16:15:49 UTC] PHP Warning: session_start(): Cannot find save handler ‘memcached’ - session startup failed in /var/www/html/fog/lib/fog/user.class.php on line 505
[10-Aug-2018 16:15:52 UTC] PHP Warning: session_destroy(): Trying to destroy uninitialized session in /var/www/html/fog/lib/fog/user.class.php on line 503
[10-Aug-2018 16:15:52 UTC] PHP Warning: session_start(): Cannot find save handler ‘memcached’ - session startup failed in /var/www/html/fog/lib/fog/user.class.php on line 505
"
I can see login screen only, but I’m not able to login
when I switch to
session.save_handler] = memcache
page is loading, but there is no any improvement. I still have to wait 5-10 secs to open e.g. HOST details or tasked snappins

george1421 · Aug 10, 2018, 7:36 PM

@andrewg78 At this time don’t enable memcached, The developers have decided to not add this feature at the moment.

Actually you should not need to follow any of this post since most of it has already been included in FOG 1.5.2 base configuration and later.

Can php-fpm make fog web-gui fast

158

12.0k

17.3k

155.2k