Fog Client service stops responding
-
@Joe-Schmitt said in Fog Client service stops responding:
@tonytonyistony while the client is in this bad state, (continually logging
Could not parse data), could you try going to the host in the web portal and clickingReset Encryption Data? Its not needed, but I want to see if it’ll force the client to re-authenticate and fix this issue itself.This is the log output after hitting the “Reset Encryption Data” button in the server.
I did not do anything else but cat the “fog.log” file. Is there anything else you wanted me to do on the client side after hitting the reset button on the server?#Fog.log ------------------------------------------------------------------------------ ----------------------------------UserTracker--------------------------------- ------------------------------------------------------------------------------ 4/5/2017 9:58 AM Client-Info Client Version: 0.11.8 4/5/2017 9:58 AM Client-Info Client OS: Linux 4/5/2017 9:58 AM Client-Info Server Version: 689680 4/5/2017 9:58 AM Middleware::Response ERROR: Unable to get subsection 4/5/2017 9:58 AM Middleware::Response ERROR: Object reference not set to an instance of an object 4/5/2017 9:58 AM Service Sleeping for 125 seconds 4/5/2017 10:01 AM Middleware::Communication URL: http://10.10.100.252/fog/management/index.php?sub=requestClientInfo&configure&newService&json 4/5/2017 10:01 AM Middleware::Response Success 4/5/2017 10:01 AM Middleware::Communication Download: http://10.10.100.252/fog/management/other/fogimg.png 4/5/2017 10:01 AM Middleware::Communication URL: http://10.10.100.252/fog/management/index.php?sub=requestClientInfo&mac=C4:6E:1F:04:54:27|B8:AC:6F:85:32:6D|52:54:00:71:D1:FE|52:54:00:71:D1:FE|B8:AC:6F:85:32:6D|52:54:00:9F:A0:77|52:54:00:2D:A7:62|52:54:00:2D:A7:62|FE:00:DD:C0:A1:06|FE:54:00:D0:9F:64|FE:00:DD:C0:A1:65|FE:54:00:7C:BC:97|FE:06:0A:0E:A1:00|FE:06:0A:0E:A1:01|FE:06:0A:0E:A1:02|FE:06:0A:0E:A1:03|52:54:00:51:61:E8|52:54:00:51:61:E8|FE:00:DD:C0:B1:07|FE:54:00:86:86:3E|FE:00:DD:C0:B1:66|FE:54:00:FB:99:70|FE:06:0A:0E:B1:00|FE:06:0A:0E:B1:01|FE:06:0A:0E:B1:02|FE:06:0A:0E:B1:03|FE:06:0A:0E:B1:04|FE:06:0A:0E:B1:05|52:54:00:A7:08:1F|52:54:00:A7:08:1F|FE:00:DD:C0:C1:08|FE:54:00:98:34:A7|FE:00:DD:C0:C1:67|FE:54:00:1E:E7:85|FE:06:0A:0E:C1:00|FE:06:0A:0E:C1:01|FE:06:0A:0E:C1:02|FE:06:0A:0E:C1:03|FE:06:0A:0E:C1:04|FE:06:0A:0E:C1:05|52:54:00:C2:F6:BD|52:54:00:C2:F6:BD|FE:00:DD:C0:D1:09|FE:54:00:A2:37:BE|FE:00:DD:C0:D1:68|FE:54:00:0A:C1:A7|FE:06:0A:0E:D1:00|FE:06:0A:0E:D1:01|FE:06:0A:0E:D1:02|FE:06:0A:0E:D1:03|FE:54:00:A8:95:EB|FE:54:00:53:C0:10|FE:54:00:AD:1C:98|FE:54:00:1A:59:4F|FE:54:00:43:73:5C|FE:54:00:F3:69:66|FE:06:0A:0E:A1:02|FE:06:0A:0E:A1:03|FE:06:0A:0E:D1:03|FE:06:0A:0E:D1:02|FE:06:0A:0E:A1:00|FE:06:0A:0E:A1:01|FE:06:0A:0E:B1:04|FE:06:0A:0E:B1:05|FE:06:0A:0E:C1:00|FE:06:0A:0E:C1:01|FE:06:0A:0E:B1:02|FE:06:0A:0E:C1:02&newService&json 4/5/2017 10:01 AM Middleware::Response ERROR: Could not parse data 4/5/2017 10:01 AM Middleware::Response ERROR: Unexpected character encountered while parsing value: T. Path '', line 0, position 0. 4/5/2017 10:01 AM Middleware::Communication URL: http://10.10.100.252/fog/service/getversion.php?clientver&newService&json 4/5/2017 10:01 AM Middleware::Communication URL: http://10.10.100.252/fog/service/getversion.php?newService&json 4/5/2017 10:01 AM Service ERROR: Unable to get cycle data 4/5/2017 10:01 AM Service ERROR: Object reference not set to an instance of an object 4/5/2017 10:01 AM Middleware::Response Success ------------------------------------------------------------------------------ ---------------------------------ClientUpdater-------------------------------- ------------------------------------------------------------------------------ 4/5/2017 10:01 AM Client-Info Client Version: 0.11.8 4/5/2017 10:01 AM Client-Info Client OS: Linux 4/5/2017 10:01 AM Client-Info Server Version: 689680 4/5/2017 10:01 AM Middleware::Response ERROR: Unable to get subsection 4/5/2017 10:01 AM Middleware::Response ERROR: Object reference not set to an instance of an object ------------------------------------------------------------------------------ ----------------------------------TaskReboot---------------------------------- ------------------------------------------------------------------------------ 4/5/2017 10:01 AM Client-Info Client Version: 0.11.8 4/5/2017 10:01 AM Client-Info Client OS: Linux 4/5/2017 10:01 AM Client-Info Server Version: 689680 4/5/2017 10:01 AM Middleware::Response ERROR: Unable to get subsection 4/5/2017 10:01 AM Middleware::Response ERROR: Object reference not set to an instance of an object ------------------------------------------------------------------------------ --------------------------------HostnameChanger------------------------------- ------------------------------------------------------------------------------ 4/5/2017 10:01 AM Client-Info Client Version: 0.11.8 4/5/2017 10:01 AM Client-Info Client OS: Linux 4/5/2017 10:01 AM Client-Info Server Version: 689680 4/5/2017 10:01 AM Middleware::Response ERROR: Unable to get subsection 4/5/2017 10:01 AM Middleware::Response ERROR: Object reference not set to an instance of an object ------------------------------------------------------------------------------ ---------------------------------SnapinClient--------------------------------- ------------------------------------------------------------------------------ 4/5/2017 10:01 AM Client-Info Client Version: 0.11.8 4/5/2017 10:01 AM Client-Info Client OS: Linux 4/5/2017 10:01 AM Client-Info Server Version: 689680 4/5/2017 10:01 AM Middleware::Response ERROR: Unable to get subsection 4/5/2017 10:01 AM Middleware::Response ERROR: Object reference not set to an instance of an object ------------------------------------------------------------------------------ --------------------------------PrinterManager-------------------------------- ------------------------------------------------------------------------------ 4/5/2017 10:01 AM Client-Info Client Version: 0.11.8 4/5/2017 10:01 AM Client-Info Client OS: Linux 4/5/2017 10:01 AM Client-Info Server Version: 689680 4/5/2017 10:01 AM Middleware::Response ERROR: Unable to get subsection 4/5/2017 10:01 AM Middleware::Response ERROR: Object reference not set to an instance of an object ------------------------------------------------------------------------------ --------------------------------PowerManagement------------------------------- ------------------------------------------------------------------------------ 4/5/2017 10:01 AM Client-Info Client Version: 0.11.8 4/5/2017 10:01 AM Client-Info Client OS: Linux 4/5/2017 10:01 AM Client-Info Server Version: 689680 4/5/2017 10:01 AM Middleware::Response ERROR: Unable to get subsection 4/5/2017 10:01 AM Middleware::Response ERROR: Object reference not set to an instance of an object ------------------------------------------------------------------------------ ----------------------------------UserTracker--------------------------------- ------------------------------------------------------------------------------ 4/5/2017 10:01 AM Client-Info Client Version: 0.11.8 4/5/2017 10:01 AM Client-Info Client OS: Linux 4/5/2017 10:01 AM Client-Info Server Version: 689680 4/5/2017 10:01 AM Middleware::Response ERROR: Unable to get subsection 4/5/2017 10:01 AM Middleware::Response ERROR: Object reference not set to an instance of an object 4/5/2017 10:01 AM Service Sleeping for 87 seconds#syslog #No new entries -
@tonytonyistony Nothing else is needed after hitting reset encryption button, the client should have automatically recovered from the failed state. I’ll send you a developer build of the client later today, if you could install it on a single problematic host that would be extremely helpful as it should log what’s causing the error. However, keep in mind developer builds of the client are extremely verbose and not meant for production use as it will likely log sensitive information.
-
@Joe-Schmitt said in Fog Client service stops responding:
but I want to see if it’ll force the client to re-authenticate and fix this issue itself.
I wanted to say this is the easiest stop gap in my opinion. I could make a cron script that detects the error, then issue some sort of restart, then I wouldn’t need user intervention to fix the issue.
-
@Joe-Schmitt said in Fog Client service stops responding:
@tonytonyistony Nothing else is needed after hitting reset encryption button, the client should have automatically recovered from the failed state. I’ll send you a developer build of the client later today, if you could install it on a single problematic host that would be extremely helpful as it should log what’s causing the error. However, keep in mind developer builds of the client are extremely verbose and not meant for production use as it will likely log sensitive information.
Awesome! Just let me know what needs to be done and Ill make it happen and get back to you.
-
@Joe-Schmitt said in Fog Client service stops responding:
@tonytonyistony Nothing else is needed after hitting reset encryption button, the client should have automatically recovered from the failed state. I’ll send you a developer build of the client later today, if you could install it on a single problematic host that would be extremely helpful as it should log what’s causing the error. However, keep in mind developer builds of the client are extremely verbose and not meant for production use as it will likely log sensitive information.
Hey Joe. So Im back at trying to figure this FOG client out in Linux. I never did receive any information from you regarding this dev build. Has there been any update or progress made getting this client working in Linux since our last discussion?
Im very confident this is due to PhP and deprecated code. Is it possible you could give me the PhP version you guys programmed in so I can try installing the right PhP version and see if that makes any positive effect?
-
@tonytonyistony said in Fog Client service stops responding:
This is the problem. The machine Im trying to image has many virtual machines running on it, which all those MAC addresses are all the interfaces being used by QEMU and VIRSH. How can I get the fog client to not send all those MAC addresses?
EDIT: When we entered the information below in a browser we got a response from JSON
Working
http://10.10.100.252/fog/management/index.php?sub=requestClientInfo&mac=C4:6E:1F:04:54:27&newService&json
JSON Response:#!ihcNot working
JSON Response:Too many MACs -
@tonytonyistony said:
When we entered the information below in a browser we got a response from JSON
Did you try the setting “Ignore MAC on client” (checkbox next to the MAC addresses) in the host settings on the FOG web GUI?
-
@Sebastian-Roth
Yes. I just did this and the client still attempts to send every single MAC address to the server. -
@tonytonyistony I am really sorry this took so long although it turns out to be really simple for you to fix. Getting to understand the client code and actually replicating the issue took me hell of a lot of time compared to the “fix”.
The
Too many MACswas a good catch of yours although I couldn’t reproduce for hours as newer versions of FOG (web UI not the client) don’t have that issue anymore and I kept looking at the wrong places. But to make a long story short, the issue for you as I see it is that theToo many MACsis not returned as JSON but as normal string which the client does not like.So there are two simple options for you:
- Set
FOG_QUICKREG_MAX_PENDING_MACSin the FOG Settings on the web UI to a value high enough for all the MACs sent by the client. I guess 100 would do you for now. Note that even if you approve(d) all the MACs for this client in the web UI the number needs to be high enough. I think this is true even in most current code and needs fixing @Tom-Elliott? - The real fix in the code is really simple as well. Just edit
/var/www/fog/lib/client/registerclient.class.phpand find this part of the code (line numbers in the front so you hopefully find it easily:
99 if (count($MACs) > $maxPending + 1) { 100 throw new Exception(_('Too many MACs')); 101 }Be aware this is in there two times, one for the new client (further up) and one for the old client (more to the bottom). You want to change only the top one! Change it and save the file:
99 if (count($MACs) > $maxPending + 1) { 100 return array('error' => _('Too many MACs')); 101 }No need to restart anything. Even your FOG client should recover on its own on the next request!!
- Set
-
@tonytonyistony Any news on this one?