Apache2 restart failed after trunk update
-
Any ideas guys? OR should I just roll back?
-
For now I have rolled back to the working version. Let me know if anyone has any ideas.
@Wayne-Workman I have now made a backup for those files we were discussing.
-
Update, so I rolled back now the system is forcing client to update to client version 0.9.11 and breaking the whole system, anyone out there that can help would be greatly appreciated.
-
This is the log from a client installed fresh with 0.9.7 and service turned on:
3/4/2016 7:39 AM RegistryHandler 64 bit registry detected 3/4/2016 7:39 AM Middleware::Communication URL: http://fog-server/fog/management/other/ca.cert.der 3/4/2016 7:40 AM Bus Became bus server 3/4/2016 7:40 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"load\"\r\n}" } 3/4/2016 7:40 AM Bus Emmiting message on channel: Status ------------------------------------------------------------------------------ --------------------------------Authentication-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:40 AM Client-Info Version: 0.9.7 3/4/2016 7:40 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 7:40 AM Data::RSA FOG Server CA cert found 3/4/2016 7:40 AM Middleware::Authentication Cert OK 3/4/2016 7:40 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 7:40 AM Middleware::Communication Unknown Response: Private key not found 3/4/2016 7:40 AM Bus Registering ParseBus in channel Power ------------------------------------------------------------------------------ ---------------------------------ClientUpdater-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:40 AM Client-Info Version: 0.9.7 3/4/2016 7:40 AM ClientUpdater Running... 3/4/2016 7:40 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=clientupdater&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:40 AM Middleware::Communication Response: Success 3/4/2016 7:40 AM Middleware::Communication URL: http://fog-server/fog/service/getversion.php?client&newService=1 3/4/2016 7:40 AM Middleware::Communication URL: http://fog-server/fog/client/FOGService.msi 3/4/2016 7:40 AM Data::RSA FOG Project cert found 3/4/2016 7:40 AM ClientUpdater Update file is authentic ------------------------------------------------------------------------------ 3/4/2016 7:41 AM Bus { "channel": "Update", "data": "{\r\n \"action\": \"start\"\r\n}" } 3/4/2016 7:41 AM Bus Emmiting message on channel: Update 3/4/2016 7:41 AM Service-Update Spawning update helper 3/4/2016 7:41 AM Bus Registering ParseBus in channel Power 3/4/2016 7:41 AM Update Helper Shutting down service... 3/4/2016 7:41 AM Service Stop requested 3/4/2016 7:41 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"unload\"\r\n}" } 3/4/2016 7:41 AM Bus Emmiting message on channel: Status 3/4/2016 7:41 AM Update Helper Killing remaining FOG processes... 3/4/2016 7:41 AM Update Helper Applying MSI... 3/4/2016 7:41 AM Update Helper --> msiexec /i "C:\Program Files (x86)\FOG\tmp\FOGService.msi" /quiet USETRAY="" HTTPS="0" WEBADDRESS="fog-server" WEBROOT="/fog" ROOTLOG="1" 3/4/2016 7:41 AM RegistryHandler 64 bit registry detected 3/4/2016 7:41 AM Middleware::Communication URL: http://fog-server/fog/management/other/ca.cert.der 3/4/2016 7:41 AM Update Helper Starting service... 3/4/2016 7:41 AM Bus Became bus server 3/4/2016 7:41 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"load\"\r\n}" } 3/4/2016 7:41 AM Bus Emmiting message on channel: Status ------------------------------------------------------------------------------ --------------------------------Authentication-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:41 AM Client-Info Version: 0.9.11 3/4/2016 7:41 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 7:41 AM Data::RSA FOG Server CA cert found 3/4/2016 7:41 AM Middleware::Authentication Cert OK 3/4/2016 7:41 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 7:41 AM Middleware::Communication Unknown Response: Private key not found 3/4/2016 7:41 AM Service Sleeping for 120 seconds```
-
This is with the client update service turned off on the client, still getting cert errors:
3/4/2016 7:46 AM RegistryHandler 64 bit registry detected 3/4/2016 7:46 AM Middleware::Communication URL: http://fog-server/fog/management/other/ca.cert.der 3/4/2016 7:47 AM Bus Became bus server 3/4/2016 7:47 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"load\"\r\n}" } 3/4/2016 7:47 AM Bus Emmiting message on channel: Status ------------------------------------------------------------------------------ --------------------------------Authentication-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 7:47 AM Data::RSA FOG Server CA cert found 3/4/2016 7:47 AM Middleware::Authentication Cert OK 3/4/2016 7:47 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 7:47 AM Middleware::Communication Unknown Response: Private key not found 3/4/2016 7:47 AM Bus Registering ParseBus in channel Power ------------------------------------------------------------------------------ ---------------------------------ClientUpdater-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM ClientUpdater Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=clientupdater&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Module is disabled on the host ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ----------------------------------TaskReboot---------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM TaskReboot Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=taskreboot&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Success 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/jobs.php?mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Invalid host certificate 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 7:47 AM Data::RSA FOG Server CA cert found 3/4/2016 7:47 AM Middleware::Authentication Cert OK 3/4/2016 7:47 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 7:47 AM Middleware::Communication Unknown Response: Private key not found ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ --------------------------------HostnameChanger------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM HostnameChanger Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=hostnamechanger&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Success 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/hostname.php?moduleid=hostnamechanger&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Invalid host certificate 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 7:47 AM Data::RSA FOG Server CA cert found 3/4/2016 7:47 AM Middleware::Authentication Cert OK 3/4/2016 7:47 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 7:47 AM Middleware::Communication Unknown Response: Private key not found ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ---------------------------------SnapinClient--------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM SnapinClient Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=snapinclient&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Success 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/snapins.checkin.php?mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: No snapins ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ --------------------------------PrinterManager-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM PrinterManager Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=printermanager&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Module is disabled globally on the FOG server ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ -----------------------------------GreenFOG----------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM GreenFOG Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=greenfog&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Module is disabled globally on the FOG server ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ----------------------------------UserTracker--------------------------------- ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Client-Info Version: 0.9.7 3/4/2016 7:47 AM UserTracker Running... 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=usertracker&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Success 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/service/usertracking.report.php?action=login&user=NASDCV80002\Administrator&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 ------------------------------------------------------------------------------ 3/4/2016 7:47 AM Middleware::Communication URL: http://fog-server/fog/management/index.php?node=client&sub=configure&newService=1 3/4/2016 7:47 AM Middleware::Communication Response: Success 3/4/2016 7:47 AM Service Sleeping for 60 seconds
-
@Tom Elliott help me figure out the issues I was having I will let him explain as I am little confused still but I get the general idea of it
-
Broke again, it is the client that is breaking it for whatever reason. I uninstalled the client rebooted and reinstalled. Once the client starts talking even though it is at 0.9.11 it starts updating it again. Here is the result:
Windows PowerShell Copyright (C) 2013 Microsoft Corporation. All rights reserved. PS C:\Users\Administrator> get-content -wait C:\fog.log.log get-content : Cannot find path 'C:\fog.log.log' because it does not exist. At line:1 char:1 + get-content -wait C:\fog.log.log + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (C:\fog.log.log:String) [Get-Content], ItemNotFoundException + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand PS C:\Users\Administrator> get-content -wait C:\fog.log 3/4/2016 8:30 AM RegistryHandler 64 bit registry detected 3/4/2016 8:30 AM Middleware::Communication URL: http://fog-server/fog/management/other/ca.cert.der 3/4/2016 8:33 AM Bus Became bus server 3/4/2016 8:33 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"load\"\r\n}" } 3/4/2016 8:33 AM Bus Emmiting message on channel: Status ------------------------------------------------------------------------------ --------------------------------Authentication-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 8:33 AM Client-Info Version: 0.9.11 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 8:33 AM Data::RSA FOG Server CA cert found 3/4/2016 8:33 AM Middleware::Authentication Cert OK 3/4/2016 8:33 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 8:33 AM Middleware::Communication Response: Success 3/4/2016 8:33 AM Middleware::Authentication Authenticated 3/4/2016 8:33 AM Bus Registering ParseBus in channel Power ------------------------------------------------------------------------------ ---------------------------------ClientUpdater-------------------------------- ------------------------------------------------------------------------------ 3/4/2016 8:33 AM Client-Info Version: 0.9.11 3/4/2016 8:33 AM ClientUpdater Running... 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=client updater&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 8:33 AM Middleware::Communication Response: Success 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/service/getversion.php?client&newService=1 ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ----------------------------------TaskReboot---------------------------------- ------------------------------------------------------------------------------ 3/4/2016 8:33 AM Client-Info Version: 0.9.11 3/4/2016 8:33 AM TaskReboot Running... 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/service/servicemodule-active.php?moduleid=taskre boot&mac=00:50:56:86:1F:C7||00:00:00:00:00:00:00:E0&newService=1 3/4/2016 8:33 AM Middleware::Communication Response: Success 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/service/jobs.php?mac=00:50:56:86:1F:C7||00:00:00 :00:00:00:00:E0&newService=1 3/4/2016 8:33 AM Middleware::Communication Response: Invalid host certificate 3/4/2016 8:33 AM Middleware::Communication URL: http://fog-server/fog/management/other/ssl/srvpublic.crt 3/4/2016 8:33 AM Data::RSA FOG Server CA cert found 3/4/2016 8:33 AM Middleware::Authentication Cert OK 3/4/2016 8:33 AM Middleware::Communication POST URL: http://fog-server/fog/management/index.php?sub=authorize 3/4/2016 8:33 AM Service Stop requested 3/4/2016 8:33 AM Bus { "channel": "Status", "data": "{\r\n \"action\": \"unload\"\r\n}" } 3/4/2016 8:33 AM Bus Emmiting message on channel: Status 3/4/2016 8:33 AM Middleware::Communication ERROR: Failed to POST data 3/4/2016 8:33 AM Middleware::Communication ERROR: Thread was being aborted. 3/4/2016 8:33 AM Middleware::Authentication ERROR: Could not authenticate 3/4/2016 8:33 AM Middleware::Authentication ERROR: Thread was being aborted. 3/4/2016 8:33 AM Middleware::Communication ERROR: Could not contact FOG server 3/4/2016 8:33 AM Middleware::Communication ERROR: Thread was being aborted. 3/4/2016 8:33 AM Service ERROR: Thread was being aborted.
-
@jbob IF the client is installed directly from the server why is it trying to update again once the service starts. That is what is breaking it that I can see. I might be wrong though.
-
I keep resetting Encryption data but it isn’t helping at all. Now none of the client will work. Any ideas guys? I know it has something to do with that because if I do it, it will work for one rotation but then it just breaks and start sending invalid host certificate again.
-
We figured out the problem. It was heavily getting spammed.
THe load average was at around 53 1 min (30 5 min) (28 15 min) on a 2 core system.
My guess (if I had one to give) was the Security tokens were colliding causing constant rechecks. I shutdown the apache2 service, cleared the security tokens of ALL hosts and restarted. It appears to have helped tremendously.
-
@Tom-Elliott Still running like a champ. So I do greatly appreciate it. You allowed me to go home and not have ot manually dis-join and rejoin every machine in the building by hand