Particlone Fails to Start
-
[Mon Jul 18 07:39:37.212110 2016] [mpm_prefork:notice] [pid 1437] AH00163: Apache/2.4.7 (Ubuntu) PHP/5.6.23-1+deprecated+dontuse+deb.sury.org~trusty+1 OpenSSL/1.0.1f c$ [Mon Jul 18 07:39:37.212138 2016] [core:notice] [pid 1437] AH00094: Command line: '/usr/sbin/apache2' [Mon Jul 18 13:56:08.991155 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit K in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991183 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit - in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991190 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit Y in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991196 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit H in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991203 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit J in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991209 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit - in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991215 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit T in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991221 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit H in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991227 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit X in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991233 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit - in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991239 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit W in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991245 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit H in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991251 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit H in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991257 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit - in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991271 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit W in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991277 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit T in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 13:56:08.991283 2016] [:error] [pid 6324] [client 172.16.19.142:49775] PHP Warning: pack(): Type H: illegal hex digit T in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.523933 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit K in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.523961 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit - in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.523969 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit Y in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.523976 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit H in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.523982 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit J in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.523988 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit - in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.523994 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit T in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.524001 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit H in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.524007 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit X in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.524013 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit - in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.524019 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit W in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.524026 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit H in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.524038 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit H in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.524045 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit - in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.524051 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit W in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.524056 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit T in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:24:56.524062 2016] [:error] [pid 5148] [client 172.16.19.149:49681] PHP Warning: pack(): Type H: illegal hex digit T in /var/www/html/fog/lib/fog/fogba$ [Mon Jul 18 14:48:10.311780 2016] [mpm_prefork:notice] [pid 1437] AH00169: caught SIGTERM, shutting down [Mon Jul 18 14:48:11.358054 2016] [mpm_prefork:notice] [pid 9188] AH00163: Apache/2.4.7 (Ubuntu) PHP/5.6.23-1+deprecated+dontuse+deb.sury.org~trusty+1 OpenSSL/1.0.1f c$ [Mon Jul 18 14:48:11.358098 2016] [core:notice] [pid 9188] AH00094: Command line: '/usr/sbin/apache2'
-
[Thu Jul 21 12:46:53.589163 2016] [:error] [pid 1624] [client 10.72.129.4:59852] PHP Warning: Packets out of order. Expected 1 received 0. Packet size=30 in /var/www/html/fog/lib/db/pdodb.class.php on line 32, referer: http://10.72.3.50/fog/management/index.php?node=home
I think this is what is happening
Could be wrong, but def an error
It will start from Server at office, but not at node, which is the log I was showing was multicast from node -
Any thoughts on this?
-
Same on this end. I was showing log on Node.
-
Are you using location plugin?
-
Not that I’m aware of.
-
@Tom-Elliott I can confirm location plugin in use on my end
-
No new revelations on this front?
I’ll be gone until Monday… I’ve been able to do unicasting but I’d love to get multicasting working because I’m way behind. Thanks!
Joe
-
@Joe-Gill I’m not one who wants to say revert, but I’m told svn version 5887 seems to work okay for Multicast. I imagine/hope this would work for you too.
To install:
svn co -r 5887 https://svn.code.sf.net/p/freeghost/code/trunk cd trunk/bin ./installfog.sh -y
See if that works, if so, stay there for now.
-
I totally understand. I’ll try that here on Monday. It’s too close to the end of the work day today to do that. Thanks!! I’ll keep you posted!
-
@Joe-Gill I believe I found the issue why your nodes weren’t continually running the service. This was completely unrelated to multicast in and of itself. This should be fixed for RC-2.
For multicast tasks, I believe I’ve found an even keel for things. The location plugin moves hosts to the node to be used. However, if the node was not the “master” node in the past, the host wouldn’t have been looking to the location’s node for its tasking. (This is not specific to you @Joe-Gill rather more to @Hanz and others using location plugin). I’ve created a hook for the FOGMulticastManager service to enable Storage Node Specific locations to start multicast tasks.
All of this should be available fog RC-2 when it get’s published.
Thanks for reporting, and sorry it took so long to respond.
-
Thanks Tom! I have updated to RC-2. I’ll let you know if the problem continues. I’ll be doing more imaging here in the next couple days.
Cheers!
-
Well, I tried running a Multi-task this evening and it hung at the Partclone screen. I did not reset the database prior to trying the Multi-task. I’ll be around tomorrow from
7 AM - 5:30 PM MST. I’m going to update/upgrade to the latest SVN release tonight. But this post is after updating to Running Version 1.3.0-RC-2 SVN Revision: 5937.Cheers,
Joe
-
Well I managed to get this working again…
Here is what I ended up doing… I upgraded to latest Trunk release (Running Version 1.3.0-RC-4 SVN Revision: 5941) and re-tried doing my multi task… It would not even start a partclone session. The log showed that no tasks were scheduled. It did, however, refresh better this time.
I stopped and started the service on both the Fog Node and the Fog Server. Than I cleared the mysql database. Voila! Things are happy!
I did notice that the multitask log file is logging things better now. I’ll be using this the rest of the week pretty heavily. I’ll keep you posted if things change!
Thanks Tom!!
-
This still seems to haunt me…
Same deal as before… Multicast seems to fail to start. Resetting things and voila, magic. I’m currently on RC-8…
Here is the latest apache2 log file…
root@Fog:/var/log/apache2# tail error.log [Mon Aug 22 08:28:33.797901 2016] [:error] [pid 1455] [client 172.16.16.150:50548] PHP Warning: array_map(): Argument #2 should be an array in /var/www/html/fog/lib/pages/dashboardpage.class.php on line 109, referer: http://172.16.1.17/fog/management/index.php [Mon Aug 22 08:28:33.797930 2016] [:error] [pid 1455] [client 172.16.16.150:50548] PHP Warning: array_combine() expects parameter 2 to be array, null given in /var/www/html/fog/lib/pages/dashboardpage.class.php on line 110, referer: http://172.16.1.17/fog/management/index.php [Mon Aug 22 08:28:34.807205 2016] [:error] [pid 1457] [client 172.16.16.150:50563] PHP Warning: array_map(): Argument #2 should be an array in /var/www/html/fog/lib/pages/dashboardpage.class.php on line 109, referer: http://172.16.1.17/fog/management/index.php [Mon Aug 22 08:28:34.807232 2016] [:error] [pid 1457] [client 172.16.16.150:50563] PHP Warning: array_combine() expects parameter 2 to be array, null given in /var/www/html/fog/lib/pages/dashboardpage.class.php on line 110, referer: http://172.16.1.17/fog/management/index.php [Mon Aug 22 13:27:27.089789 2016] [:error] [pid 1457] [client 172.16.17.122:42012] PHP Warning: ftp_delete(): Delete operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 59 [Mon Aug 22 13:27:27.090008 2016] [:error] [pid 1457] [client 172.16.17.122:42012] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 173 [Mon Aug 22 13:27:27.103107 2016] [:error] [pid 1457] [client 172.16.17.122:42012] PHP Warning: ftp_delete(): Delete operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 59 [Mon Aug 22 13:45:30.936479 2016] [:error] [pid 1456] [client 172.16.16.150:61835] PHP Strict Standards: Only variables should be passed by reference in /var/www/html/fog/lib/fog/group.class.php on line 66, referer: http://172.16.1.17/fog/management/index.php?node=group&sub=edit&id=30 [Mon Aug 22 13:45:48.506431 2016] [:error] [pid 1455] [client 172.16.16.150:61854] PHP Strict Standards: Only variables should be passed by reference in /var/www/html/fog/lib/fog/group.class.php on line 78, referer: http://172.16.1.17/fog/management/index.php?node=group&sub=edit&id=30 [Mon Aug 22 13:46:53.879293 2016] [:error] [pid 2767] [client 172.16.16.150:61922] PHP Warning: array_walk() expects parameter 1 to be array, null given in /var/www/html/fog/lib/fog/group.class.php on line 214, referer: http://172.16.1.17/fog/management/index.php?node=group&sub=deploy&id=30&type=8
-
@Joe-Gill make sure that both the server and node (via logviewer–>multicast show that multicast session is actually tasked…I’ve had to cancel both active multicast task and active tasks, along with restarting FOGMulticastManager on both server and node, after killing mysql (TASK=8). Restart multicast tasking and it should start right up. I’ve even done a travel node, and can swear to it working. The key is making sure the logviewer multicast from server and node, show same tasking.
-
@Hanz I noticed today that the neither the node or the server showed the task was multi-tasked. The funny thing is that I’ve had this happen so frequently that it’s almost easier to just go in and clear the database and reset things before I even start things. To have to go to each machine and manually shut things down becomes a bit of a pain.
I appreciate the advice!
Cheers!
-
@Joe-Gill before you start the machines into pxe boot, make sure the task is showing on node will alleviate having to shut them all down each time. Set your Multicast timeout high enough to give you time…mine is set for 5 minutes.
-
@Hanz Can you elaborate more on that last point? I think that is highly interesting, and a vital piece of knowledge.
-
@Wayne-Workman
"“make sure that both the server and node (via logviewer–>multicast show that multicast session is actually tasked…I’ve had to cancel both active multicast task and active tasks, along with restarting FOGMulticastManager on both server and node, after killing mysql (TASK=8). Restart multicast tasking and it should start right up. I’ve even done a travel node, and can swear to it working. The key is making sure the logviewer multicast from server and node, show same tasking”That’s all I got, not sure how to elaborate much more.