Posts made by RipAU

Awesome, thanks.

Also just FYI after doing a fresh install of CentOS 7 that has been updated with the latest SVN version of Fog Version: 7332
I’m also getting the same errors from httpd

[Wed Apr 27 13:30:18.351161 2016] [:error] [pid 12236] [client 10.254.14.101:53436] PHP Warning:  session_destroy(): Trying to destroy uninitialized session in /var/www/html/fog/lib/fog/user.class.php on line 113
[Wed Apr 27 13:30:28.607672 2016] [:error] [pid 9846] [client 10.254.14.101:53470] PHP Warning:  implode(): Invalid arguments passed in /var/www/html/fog/lib/fog/eventmanager.class.php on line 67
[Wed Apr 27 13:30:28.665747 2016] [:error] [pid 9846] [client 10.254.14.101:53470] PHP Warning:  session_destroy(): Trying to destroy uninitialized session in /var/www/html/fog/lib/fog/user.class.php on line 113
[Wed Apr 27 13:30:38.938836 2016] [:error] [pid 12215] [client 10.254.14.101:53507] PHP Warning:  implode(): Invalid arguments passed in /var/www/html/fog/lib/fog/eventmanager.class.php on line 67
[Wed Apr 27 13:30:38.996816 2016] [:error] [pid 12215] [client 10.254.14.101:53507] PHP Warning:  session_destroy(): Trying to destroy uninitialized session in /var/www/html/fog/lib/fog/user.class.php on line 113
[Wed Apr 27 13:30:49.257933 2016] [:error] [pid 9864] [client 10.254.14.101:53541] PHP Warning:  implode(): Invalid arguments passed in /var/www/html/fog/lib/fog/eventmanager.class.php on line 67
[Wed Apr 27 13:30:49.316223 2016] [:error] [pid 9864] [client 10.254.14.101:53541] PHP Warning:  session_destroy(): Trying to destroy uninitialized session in /var/www/html/fog/lib/fog/user.class.php on line 113

Just tested again with a full update from the working version.

I’m still convinced it is an issue with the update in FOG with my server as the server load is so much lower before the update.

last hit: 01:32:48         atop runtime:  0 days, 00:00:35             01:32:49
All:         2351 reqs (  67.2/sec)       1470.0K (   42.0K/sec)     640.3B/req
2xx:    2351 ( 100%) 3xx:       0 ( 0.0%) 4xx:     0 ( 0.0%) 5xx:     0 ( 0.0%)
R ( 30s):    2018 reqs (  67.3/sec)       1265.5K (   42.2K/sec)     642.2B/req
2xx:    2018 ( 100%) 3xx:       0 ( 0.0%) 4xx:     0 ( 0.0%) 5xx:     0 ( 0.0%)

 REQS REQ/S    KB KB/S URL
  658 21.93 205.1  6.8*/fog/management/index.php
  644 21.47  1056 35.2 /fog/management/other/ssl/srvpublic.crt
  566 18.87   2.8  0.1 /fog/service/jobs.php
   79  2.63   0.4  0.0 /fog/service/greenfog.php
   46  1.53   0.2  0.0 /fog/service/servicemodule-active.php
   23  0.79   1.1  0.0 /fog/service/Printers.php
    2  0.10   0.0  0.0 *

Also from mysql SHOW FULL PROCESSLIST;

MariaDB [(none)]> SHOW FULL PROCESSLIST;
+-----+------+-----------------+------+---------+------+-------+-----------------------+----------+
| Id  | User | Host            | db   | Command | Time | State | Info                  | Progress |
+-----+------+-----------------+------+---------+------+-------+-----------------------+----------+
|  14 | root | localhost:60778 | fog  | Sleep   |    1 |       | NULL                  |    0.000 |
|  19 | root | localhost:60790 | fog  | Sleep   |    1 |       | NULL                  |    0.000 |
|  39 | root | localhost:60856 | fog  | Sleep   |    0 |       | NULL                  |    0.000 |
|  42 | root | localhost:60862 | fog  | Sleep   |    0 |       | NULL                  |    0.000 |
|  43 | root | localhost:60890 | fog  | Sleep   |    5 |       | NULL                  |    0.000 |
|  44 | root | localhost:60902 | fog  | Sleep   |  514 |       | NULL                  |    0.000 |
|  45 | root | localhost:60914 | fog  | Sleep   |  537 |       | NULL                  |    0.000 |
|  46 | root | localhost:60950 | fog  | Sleep   |   55 |       | NULL                  |    0.000 |
|  47 | root | localhost:32782 | fog  | Sleep   |  152 |       | NULL                  |    0.000 |
|  49 | root | localhost:33290 | fog  | Sleep   |    1 |       | NULL                  |    0.000 |
|  51 | root | localhost:33474 | fog  | Sleep   |    0 |       | NULL                  |    0.000 |
|  52 | root | localhost:33914 | fog  | Sleep   |    1 |       | NULL                  |    0.000 |
|  53 | root | localhost:33924 | fog  | Sleep   |    0 |       | NULL                  |    0.000 |
|  56 | root | localhost:33968 | fog  | Sleep   |    0 |       | NULL                  |    0.000 |
|  58 | root | localhost:33984 | fog  | Sleep   |    0 |       | NULL                  |    0.000 |

Just FYI there is about 120 lines of this that is the same.

I’ve also upped the CPU and the Memory just to test but it is the same result, which is why I’m thinking its more an issue with the FOG update than 32bit vs 64bit.

As well as errors in the HTTPD log listing PHP warnings.

[Wed Apr 27 11:08:21.707362 2016] [:error] [pid 6824] [client 10.254.3.15:50021] PHP Warning:  implode(): Invalid arguments passed in /var/www/html/fog/lib/fog/eventmanager.class.php on line 67
[Wed Apr 27 11:08:21.728167 2016] [:error] [pid 7001] [client 10.254.1.171:60754] PHP Warning:  implode(): Invalid arguments passed in /var/www/html/fog/lib/fog/eventmanager.class.php on line 67

Thanks everyone so far for the help

Edit: forgot the CPU from htop

@Wayne-Workman

No worries, I’ll look at spinning up an updated CentOS 7 machine to test the SVN with.

That said, I currently have a working version of CentOS 7 64bit with 1 Virtual CPU on the same ESXi and that didn’t have any problems being installed and everything is listed as 64bit by vmware etc.

I’ll give this a go over the next few days and report back.

Cheers,

@Wayne-Workman

I’m happy to try a re-install and try to migrate the current database and images to an updated OS as I was going update to CentOS7 in future as I would like to be able to keep everything up to date.

I guess I generally will only give it a single CPU unless required as that was what I was told by a VMware engineer at a training session a few years ago and I can’t say I’ve even heard that only allocating a single 64bit capable CPU ends up running as a 32bit CPU on VMware ESXi.

@Wayne-Workman

Okay I’m confused.

So you are saying even though the ESXi host is 64bit CPU and I installed the Fedora x86_64 version of the OS
and the kernel is listed as x86_64 GNU/Linux.
Because I only allocated 1 virtual CPU It installed as 32bit?

@Wayne-Workman

I can easily up the CPU and RAM - The server is the 64bit version of Fedora.

 4.2.7-200.fc22.x86_64 #1 SMP Thu Dec 10 03:28:47 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

@Wayne-Workman
Yep no worries.

The server is ESXi 6.
At this stage it has 1vCPU has been assigned with 2GB of RAM.
Server CPUS are Intel Xenon CPU E5-2630 @ 2.4ghz

Fog Server at this stage only has about 300 hosts (as soon as everything is smoothed out I was going to migrate the rest of our hosts over and this will be about 1100)

I can revert to the updated fog version of the snapshot after It was updated and run that command.
I’ll do that now.

Cheers,

@Sebastian-Roth

Yep just did a revert back to the previous version then. It is Fog Version: 6207
The CPU usage is a little higher than I’d like (fluctuates from 20% to 65% depending on the amount of traffic to the server, which is one of the reasons I was going to update) but otherwise has been working perfectly with no real issues.
Server load generally seems fine as well - load average: 0.88, 1.74, 1.55

Hi, Guys.

As a bit of a hopeful move I updated to the latest SVN update (FOG Version: 7330)
I’ve found the web interface near unusable because of the load that is now being put on the server(Huge number of httpd processes are now being spawned taking up most of the CPU and quite allot of mysqld processes as well)

I’ll roll back to a working version of the SVN with a snapshot. Happy to try to update and test if this helps

I’ve taken several screenshots that hopefully help.

just to clarify this is with Fedora 22 Server, PHP7 from Remi Repository with FOG version 7330 on an VMware ESXi server.
Previously it was working perfectly

Cheers.

Just checked then.

I’m not sure I can see an issue? with the following.

last hit: 03:35:45         atop runtime:  0 days, 00:02:05             03:35:46
All:         2043 reqs (  16.3/sec)        467.5K ( 3829.8B/sec)     234.3B/req
2xx:    1817 (88.9%) 3xx:      52 ( 2.5%) 4xx:     2 ( 0.1%) 5xx:   172 ( 8.4%)
R ( 30s):     476 reqs (  15.9/sec)        116.9K ( 3990.2B/sec)     251.5B/req
2xx:     424 (89.1%) 3xx:       6 ( 1.3%) 4xx:     0 ( 0.0%) 5xx:    46 ( 9.7%)

 REQS REQ/S    KB KB/S URL
  144  4.80   0.7  0.0*/fog/service/servicemodule-active.php
   94  3.13   4.1  0.1 /fog/management/index.php
   66  2.20   2.4  0.1 /fog/status/bandwidth.php
   49  1.63  80.3  2.7 /fog/management/other/ssl/srvpublic.crt
   43  1.43   1.5  0.0 /fog/service/Printers.php
   14  0.48   0.1  0.0 /fog/service/greenfog.php
   13  0.43  27.5  0.9 /fog/service/printerlisting.php
   13  0.43   0.1  0.0 /fog/service/snapins.checkin.php
   11  0.37   0.1  0.0 /fog/service/hostname.php
   10  0.36   0.0  0.0 /fog/service/jobs.php
   10  0.36   0.1  0.0 /fog/service/getversion.php
    6  0.21   0.0  0.0 /fog/index.php
    3  0.10   0.0  0.0 *

looks like most requests are being sent to servicemodule-active.php ?

I decided to check some fog.log files on one of my hosts and I am also seeing this.

26/04/2016 11:27 AM Client-Info Version: 0.9.12
 26/04/2016 11:27 AM Middleware::Communication URL: http://fog2/fog/management/other/ssl/srvpublic.crt
 26/04/2016 11:27 AM Data::RSA FOG Server CA cert found
 26/04/2016 11:27 AM Middleware::Authentication Cert OK
 26/04/2016 11:27 AM Middleware::Communication POST URL: http://fog2/fog/management/index.php?sub=authorize
 26/04/2016 11:27 AM Middleware::Communication ERROR: Failed to POST data
 26/04/2016 11:27 AM Middleware::Communication ERROR: The remote server returned an error: (500) Internal Server Error.
 26/04/2016 11:27 AM Middleware::Authentication ERROR: Could not authenticate
 26/04/2016 11:27 AM Middleware::Authentication ERROR: Object reference not set to an instance of an object.
 26/04/2016 11:27 AM Service Sleeping for 120 seconds```

Ok, Just did an update to the latest trunk (in fog version its listed as 7306)
I have lost the graph that lists the amount of storage left and when clicking on the name it has access denied and wrong user name and password listed. I’m also receiving the following errors in HTTPD (different ones this time )

[Tue Apr 26 11:13:35.131109 2016] [:error] [pid 1499] [client 10.254.1.59:55287] PHP Fatal error:  Uncaught Error: Using $this when not in object context in /var/www/html/fog/lib/fog/fogpage.class.php:743\nStack trace:\n#0 /var/www/html/fog/commons/init.php(141): FOGPage::authorize()\n#1 /var/www/html/fog/commons/base.inc.php(10): require_once('/var/www/html/f...')\n#2 /var/www/html/fog/management/index.php(2): require('/var/www/html/f...')\n#3 {main}\n  thrown in /var/www/html/fog/lib/fog/fogpage.class.php on line 743

CPU usage is still hovering very high.

Thanks again for the help

Fog Version 7302

Sorry I tried having the image in the same post but it was showing up broken.

This is the current load since updating to the latest Trunk.

There are about 300 Hosts being managed by FOG at this stage on Fedora 22 with PHP 7 from the Remi Repository.
They are all using the new fog client as well. (I also reset the encryption data on all hosts after the update, this didn’t seem to change the load at all)
Server is running on VMware ESXi.

Cheers,

Ok, Just updated and did a reset on the encryption data.
We only have about 300 hosts on this version of FOG at this stage.

The CPU usage is still really high and I am now getting the following Error in the HTTPD logs.

[Tue Apr 26 09:03:40.630623 2016] [:error] [pid 1225] [client 10.254.13.91:61830] PHP Warning:  curl_setopt_array(): Array keys must be CURLOPT constants or equivalent integer values in /var/www/html/fog/lib/fog/fogurlrequests.class.php on line 99, referer: http://fog2/fog/management/index.php?node=home

This is being repeated over and over.

Perfect, I’ll update that now, thanks
This is the screenshot prior to the update. As you can see if mainly the httpd process spawning 8 or so processes at 8 or 9% CPU usage each.

I’ll let you know how it goes.

Cheers,

Hi, Guys.

This is more an FYI I guess.

I have found that after successfully updating the FOG via SVN to FOG Version: 7254
That I am seeing the following error:

[Fri Apr 22 15:25:28.814288 2016] [:error] [pid 899] [client 10.254.13.91:62916] PHP Warning:  Cannot modify header information - headers already sent by (output started at /var/www/html/fog/lib/fog/page.class.php:176) in /var/www/html/fog/lib/fog/page.class.php on line 179, referer: http://fog2/fog/management/index.php?node=report&sub=imaging-log
[Fri Apr 22 15:25:28.814349 2016] [:error] [pid 899] [client 10.254.13.91:62916] PHP Warning:  Cannot modify header information - headers already sent by (output started at /var/www/html/fog/lib/fog/page.class.php:176) in /var/www/html/fog/lib/fog/page.class.php on line 180, referer: http://fog2/fog/management/index.php?node=report&sub=imaging-log

I am on Fedora 22 with PHP7 from the remi-repo but I also saw this error on the previous php version (just updated it) which was php5 I think.

As a side note I’ve also found the CPU usage had jumped quite a bit since the previous trunk version I used (back in december It was updated)

I’m not sure all of this is related?

Thanks guys.

Ok this worked by recreating the keys. Install worked fine and server seems to be working well.
Only thing I have noticed is several labs worth of Desktops had invalid tokens so I’ve needed to “Reset Encryption Data” to get them talking back to the server again.

Aside from that all is working well

Thanks again.

After going through the forum some more I found this - ./installfog.sh --recreate-keys
Would this help fix the issue at all with the upgrade?

https://forums.fogproject.org/topic/6975/change-server-but-not-change-certificate/6

From reading this I should be able to run this without affecting the currently running clients? as it should use the current CA?

@Wayne-Workman said in Apache2 for fog vhost - failed - Trying to update Trunk:

Yes, they are. Here a while back, the locations of these files were changed. Used to be it was /opt/fog/.ssl I think?? And now it’s /opt/fog/snapins/ssl

I’m just checking now as well, I don’t have any files in /opt/fog/.ssl I only have the following structure:

/opt/fog/log
/opt/fog/service
/opt/fog/snapins
/opt/fog/utils
/opt/fog/.fogsettings

So I’m suspecting that something is going wrong with regenerating the keys from the CA.