Storage Node Vanished after upgrading to 1.5.2
-
My storage node is no longer shown under storage in the web GUI and when I try to re-add the node it says it already exists. I upgrade both the Node and the Server from 1.4.4 to .1.5.2.
OS: Ubuntu 16.04
-
Found the problem and have a fix for it. If the maxClients are set to 0, the problem was the node was returning a division by 0 (not allowed in normal math either). This caused the lookup to fail with an error.
I’ve corrected this error in the working branch if you want to install it.
cd /path/to/fogproject/git git checkout working git pull cd bin ./installfog.sh -y -
@flareimp There are likely errors in your apache error log from when you try to list the storage nodes. Can you post those errors in a code block please? They will be in
/var/log/apache2/error_log -
@wayne-workman Here is the log, I blocked out the IP Address and DNS name though.
[Mon Apr 16 07:35:01.869933 2018] [mpm_prefork:notice] [pid 2845] AH00163: Apache/2.4.33 (Ubuntu) OpenSSL/1.1.0h configured – resuming normal operations
[Mon Apr 16 07:35:01.869963 2018] [core:notice] [pid 2845] AH00094: Command line: ‘/usr/sbin/apache2’
[Mon Apr 16 10:44:13.433798 2018] [mpm_prefork:notice] [pid 2845] AH00169: caught SIGTERM, shutting down
[Mon Apr 16 10:45:25.108311 2018] [mpm_prefork:notice] [pid 2815] AH00163: Apache/2.4.33 (Ubuntu) OpenSSL/1.1.0h configured – resuming normal operations
[Mon Apr 16 10:45:25.140613 2018] [core:notice] [pid 2815] AH00094: Command line: ‘/usr/sbin/apache2’
[Mon Apr 16 10:46:37.034696 2018] [php7:warn] [pid 2823] [client 10.x.x.13:49926] PHP Warning: Division by zero in /var/www/html/fog/lib/fog/storagenode.class.php on line 315, referer: http://fogserver/fog/management/index.php
[Mon Apr 16 10:46:37.497480 2018] [php7:warn] [pid 2823] [client 10.x.x.13:49926] PHP Warning: Division by zero in /var/www/html/fog/lib/fog/storagenode.class.php on line 315, referer: http://fogserver/fog/management/index.php -
@flareimp Did you try to replicate the issue just before you checked the log? I ask because I don’t see any major issues in this log.
-
@wayne-workman After trying to re-add the node I checked the log and it was empty. So I restarted the server and that’s what was in the log.
-
Maybe tomorrow you can get with me and i can help remotely?
This would be the best bet. I’ve yet to hear of a node simply disappearing ever. I can take a look directly at the issue.
-
@tom-elliott Sure thing, just let me know when is good for you sir.
-
I had a similar issue, and rebuilt to get around it. After rebuilding the issue came back but I figured out why it happened and how to fix it. At least with my config. YMMV
I had 2 “Virtual” storage nodes to allow replication to different groups from one master node. I set these virtual nodes and the master to have 0 slots for imaging. In 1.4 and 1.5, this worked fine. In 1.5.2, this broke then nodes and they showed up in the GUI as “Node not accessible”
I then logged into the MySQL database and edited the entry for the number of hosts allowed to connect to those nodes from 0 to 1, and they popped right back up in the GUI.
Like I said, this may not be your issue at all, but it was mine and similar to the symptoms you describe.
-
I’m good to view whenever you are @FlareImp
Sorry for the delay.
-
Found the problem and have a fix for it. If the maxClients are set to 0, the problem was the node was returning a division by 0 (not allowed in normal math either). This caused the lookup to fail with an error.
I’ve corrected this error in the working branch if you want to install it.
cd /path/to/fogproject/git git checkout working git pull cd bin ./installfog.sh -y -
@tom-elliott your the man! that was the issue. My Node was set to support 0 clients. thank you all for your awesome product!
