Apache2 restart failed after trunk update
-
@Psycholiquid If it were me in your position - I would roll back to when it was working,
And I would copy
/opt/fog/snapins/CAto /root for safe keeping, I’d also copy/opt/fog/snapins/sslto /root.cp -r /opt/fog/snapins/CA /root cp -r /opt/fog/snapins/ssl /rootI know that it wasn’t long ago that all the paths and stuff changed for the CA and ssl directories - so you’ll just need to pay close attention to those after you update forwards.
This should take no time. Roll back, backup your CA and ssl folders, update to the latest. Check everything.
However it’s perfectly fine if you want to wait for further input from Tom or Jbob and I’d take zero offense from that.
-
Yeah I will wait but I agree making a backup of those is not a bad idea at all.
-
Any ideas guys? OR should I just roll back?
-
For now I have rolled back to the working version. Let me know if anyone has any ideas.
@Wayne-Workman I have now made a backup for those files we were discussing.
-
Update, so I rolled back now the system is forcing client to update to client version 0.9.11 and breaking the whole system, anyone out there that can help would be greatly appreciated.
-
This is the log from a client installed fresh with 0.9.7 and service turned on:
3/4/2016 7:39 AM RegistryHandler 64 bit registry detected 3/4/2016 7:39 AM Middleware::Communication URL: http://fog-server/fog/management/other/ca.cert.der 3/4/2016 7:40 AM Bus Became bus server 3/4/2016 7:40 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"load\"\r\n}" } 3/4/2016 7:40 AM Bus Emmiting message on channel: Status ------------------------------------------------------------------------------ --------------------------------Authentication-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:40 AM Client-Info Version: 0.9.7 3/4/2016 7:40 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 7:40 AM Data::RSA FOG Server CA cert found 3/4/2016 7:40 AM Middleware::Authentication Cert OK 3/4/2016 7:40 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 7:40 AM Middleware::Communication Unknown Response: Private key not found 3/4/2016 7:40 AM Bus Registering ParseBus in channel Power ------------------------------------------------------------------------------ ---------------------------------ClientUpdater-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:40 AM Client-Info Version: 0.9.7 3/4/2016 7:40 AM ClientUpdater Running... 3/4/2016 7:40 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=clientupdater&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:40 AM Middleware::Communication Response: Success 3/4/2016 7:40 AM Middleware::Communication URL: http://fog-server/fog/service/getversion.php?client&newService=1 3/4/2016 7:40 AM Middleware::Communication URL: http://fog-server/fog/client/FOGService.msi 3/4/2016 7:40 AM Data::RSA FOG Project cert found 3/4/2016 7:40 AM ClientUpdater Update file is authentic ------------------------------------------------------------------------------ 3/4/2016 7:41 AM Bus { "channel": "Update", "data": "{\r\n \"action\": \"start\"\r\n}" } 3/4/2016 7:41 AM Bus Emmiting message on channel: Update 3/4/2016 7:41 AM Service-Update Spawning update helper 3/4/2016 7:41 AM Bus Registering ParseBus in channel Power 3/4/2016 7:41 AM Update Helper Shutting down service... 3/4/2016 7:41 AM Service Stop requested 3/4/2016 7:41 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"unload\"\r\n}" } 3/4/2016 7:41 AM Bus Emmiting message on channel: Status 3/4/2016 7:41 AM Update Helper Killing remaining FOG processes... 3/4/2016 7:41 AM Update Helper Applying MSI... 3/4/2016 7:41 AM Update Helper --> msiexec /i "C:\Program Files (x86)\FOG\tmp\FOGService.msi" /quiet USETRAY="" HTTPS="0" WEBADDRESS="fog-server" WEBROOT="/fog" ROOTLOG="1" 3/4/2016 7:41 AM RegistryHandler 64 bit registry detected 3/4/2016 7:41 AM Middleware::Communication URL: http://fog-server/fog/management/other/ca.cert.der 3/4/2016 7:41 AM Update Helper Starting service... 3/4/2016 7:41 AM Bus Became bus server 3/4/2016 7:41 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"load\"\r\n}" } 3/4/2016 7:41 AM Bus Emmiting message on channel: Status ------------------------------------------------------------------------------ --------------------------------Authentication-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:41 AM Client-Info Version: 0.9.11 3/4/2016 7:41 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 7:41 AM Data::RSA FOG Server CA cert found 3/4/2016 7:41 AM Middleware::Authentication Cert OK 3/4/2016 7:41 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 7:41 AM Middleware::Communication Unknown Response: Private key not found 3/4/2016 7:41 AM Service Sleeping for 120 seconds``` -
This is with the client update service turned off on the client, still getting cert errors:
3/4/2016 7:46 AM RegistryHandler 64 bit registry detected 3/4/2016 7:46 AM Middleware::Communication URL: http://fog-server/fog/management/other/ca.cert.der 3/4/2016 7:47 AM Bus Became bus server 3/4/2016 7:47 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"load\"\r\n}" } 3/4/2016 7:47 AM Bus Emmiting message on channel: Status ------------------------------------------------------------------------------ --------------------------------Authentication-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 7:47 AM Data::RSA FOG Server CA cert found 3/4/2016 7:47 AM Middleware::Authentication Cert OK 3/4/2016 7:47 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 7:47 AM Middleware::Communication Unknown Response: Private key not found 3/4/2016 7:47 AM Bus Registering ParseBus in channel Power ------------------------------------------------------------------------------ ---------------------------------ClientUpdater-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM ClientUpdater Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=clientupdater&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Module is disabled on the host ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ----------------------------------TaskReboot---------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM TaskReboot Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=taskreboot&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Success 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/jobs.php?mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Invalid host certificate 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 7:47 AM Data::RSA FOG Server CA cert found 3/4/2016 7:47 AM Middleware::Authentication Cert OK 3/4/2016 7:47 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 7:47 AM Middleware::Communication Unknown Response: Private key not found ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ --------------------------------HostnameChanger------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM HostnameChanger Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=hostnamechanger&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Success 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/hostname.php?moduleid=hostnamechanger&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Invalid host certificate 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 7:47 AM Data::RSA FOG Server CA cert found 3/4/2016 7:47 AM Middleware::Authentication Cert OK 3/4/2016 7:47 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 7:47 AM Middleware::Communication Unknown Response: Private key not found ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ---------------------------------SnapinClient--------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM SnapinClient Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=snapinclient&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Success 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/snapins.checkin.php?mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: No snapins ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ --------------------------------PrinterManager-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM PrinterManager Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=printermanager&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Module is disabled globally on the FOG server ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ -----------------------------------GreenFOG----------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM GreenFOG Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=greenfog&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Module is disabled globally on the FOG server ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ----------------------------------UserTracker--------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM UserTracker Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=usertracker&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Success 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/usertracking.report.php?action=login&user=NASDCV80002\Administrator&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/management/index.php?node=client&sub=configure&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Success 3/4/2016 7:47 AM Service Sleeping for 60 seconds -
@Tom Elliott help me figure out the issues I was having I will let him explain as I am little confused still but I get the general idea of it
-
Broke again, it is the client that is breaking it for whatever reason. I uninstalled the client rebooted and reinstalled. Once the client starts talking even though it is at 0.9.11 it starts updating it again. Here is the result:
Windows PowerShell Copyright (C) 2013 Microsoft Corporation. All rights reserved. PS C:\Users\Administrator> get-content -wait C:\fog.log.log get-content : Cannot find path 'C:\fog.log.log' because it does not exist. At line:1 char:1 + get-content -wait C:\fog.log.log + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (C:\fog.log.log:String) [Get-Content], ItemNotFoundException + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand PS C:\Users\Administrator> get-content -wait C:\fog.log 3/4/2016 8:30 AM RegistryHandler 64 bit registry detected 3/4/2016 8:30 AM Middleware::Communication URL: http://fog-server/fog/management/other/ca.cert.der 3/4/2016 8:33 AM Bus Became bus server 3/4/2016 8:33 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"load\"\r\n}" } 3/4/2016 8:33 AM Bus Emmiting message on channel: Status ------------------------------------------------------------------------------ --------------------------------Authentication-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 8:33 AM Client-Info Version: 0.9.11 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 8:33 AM Data::RSA FOG Server CA cert found 3/4/2016 8:33 AM Middleware::Authentication Cert OK 3/4/2016 8:33 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 8:33 AM Middleware::Communication Response: Success 3/4/2016 8:33 AM Middleware::Authentication Authenticated 3/4/2016 8:33 AM Bus Registering ParseBus in channel Power ------------------------------------------------------------------------------ ---------------------------------ClientUpdater-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 8:33 AM Client-Info Version: 0.9.11 3/4/2016 8:33 AM ClientUpdater Running... 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=client updater&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 8:33 AM Middleware::Communication Response: Success 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/service/getversion.php?client&newService=1 ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ----------------------------------TaskReboot---------------------------------- ------------------------------------------------------------------------------ 3/4/2016 8:33 AM Client-Info Version: 0.9.11 3/4/2016 8:33 AM TaskReboot Running... 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=taskre boot&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 8:33 AM Middleware::Communication Response: Success 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/service/jobs.php?mac=00:50:56:86:1F:C7||00:00:00 :00:00:00:00:E0&newService=1 3/4/2016 8:33 AM Middleware::Communication Response: Invalid host certificate 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 8:33 AM Data::RSA FOG Server CA cert found 3/4/2016 8:33 AM Middleware::Authentication Cert OK 3/4/2016 8:33 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 8:33 AM Service Stop requested 3/4/2016 8:33 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"unload\"\r\n}" } 3/4/2016 8:33 AM Bus Emmiting message on channel: Status 3/4/2016 8:33 AM Middleware::Communication ERROR: Failed to POST data 3/4/2016 8:33 AM Middleware::Communication ERROR: Thread was being aborted. 3/4/2016 8:33 AM Middleware::Authentication ERROR: Could not authenticate 3/4/2016 8:33 AM Middleware::Authentication ERROR: Thread was being aborted. 3/4/2016 8:33 AM Middleware::Communication ERROR: Could not contact FOG server 3/4/2016 8:33 AM Middleware::Communication ERROR: Thread was being aborted. 3/4/2016 8:33 AM Service ERROR: Thread was being aborted. -
@jbob IF the client is installed directly from the server why is it trying to update again once the service starts. That is what is breaking it that I can see. I might be wrong though.
-
I keep resetting Encryption data but it isn’t helping at all. Now none of the client will work. Any ideas guys? I know it has something to do with that because if I do it, it will work for one rotation but then it just breaks and start sending invalid host certificate again.
-
We figured out the problem. It was heavily getting spammed.
THe load average was at around 53 1 min (30 5 min) (28 15 min) on a 2 core system.
My guess (if I had one to give) was the Security tokens were colliding causing constant rechecks. I shutdown the apache2 service, cleared the security tokens of ALL hosts and restarted. It appears to have helped tremendously.
Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.
Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)
Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG
-
@Tom-Elliott Still running like a champ. So I do greatly appreciate it. You allowed me to go home and not have ot manually dis-join and rejoin every machine in the building by hand
