FOG 1.5.5 SLOW OPEN STORAGES NODES

[wanderson]

@george1421

only the 1.5.5 server is ratified, the main server is in production in version 1.4.0 and uses the 35 storage nodes in version 1.4.0 and is working perfectly.

I just imported the FOG version 1.4.0 server database into the FOG server 1.5.5 environment to see how it would behave with the database import and whether it would maintain communication with the storage nodes .

[george1421]

@wanderson Ok I understand a bit better how you have things setup. As long as the 1.5.5 sever doesn’t have any images to push then you can run mixed.

I can tell you fog 1.5.x is very much different than fog 1.4.x as windows 10 is very much different than windows 7. While both 1.4.x and 1.5.x is FOG, they are different in how they manage things internally. There is a bigger demand on system resources in FOG 1.5.x over 1.4.x. Look at the top command to see if you need to add more vCPU or memory to your 1.5.5 fog server.

[wanderson]

@george1421

[Sebastian Roth]

@wanderson This issue is not specific to 1.5.5 but more 1.5.x and has to do with master node trying to contact the storage nodes when you click the listing. I am working on this part and hopefully have a solution to it soon.

[wanderson]

@Sebastian-Roth

True, I had tried to migrate to version 1.5.0 and I got this problem I gave up, then tried again in version 1.5.4 and the problem still persisted, now I saw that version 1.5.5 was out and the problem is still happening, but I am happy that you understand that there is this problem for those who use many nodes, I hope that I can solve and I know that you will get … thanks for the help.

[Sebastian Roth]

@wanderson The issue as I understand it so far is not that much about many storage nodes but about the nodes taking a long time (timeout) when trying to talk to each other. Please do me a favor and open the FOG configuration page http://ip.of.fog.srv/fog/management/index.php?node=about and post a screenshot of that here. What I hope to see is if all the storage nodes show proper version info. Mind you this will take a long time as well, and could possibly timeout in the end (Gateway error).

[george1421]

@Sebastian-Roth said in FOG 1.5.5 SLOW OPEN STORAGES NODES:

…and could possibly timeout in the end (Gateway error).

If we get this error there is a apache config file change we can make to extend the timeout to what ever length we need.

ref: https://forums.fogproject.org/topic/12573/intermittent-error-504/10

[Sebastian Roth]

@george1421 Nice idea but I’d better make our code to not wait for such a long time!

@wanderson I was now able to replicate the issue and still hope that we can fix this in the code of the web UI. As a quick fix you need to check all your storage node configs (/var/www/html/fog/lib/fog/config.class.php). I am fairly sure at least one of your storage nodes has the wrong database host (or user/pass) set which is causing the very long delay. They should all point to your main FOG server.

[wanderson]

@Sebastian-Roth But the 35 nodes using version 1.4.0 do not give this problem, only in version 1.5.X

[wanderson]

@Sebastian-Roth

Watch ----> https://youtu.be/EmL0gVIMbEI

[wanderson]

@george1421

Watch ----> https://youtu.be/EmL0gVIMbEI

[Sebastian Roth]

@wanderson Thanks for the video again. You are right, 1.4.x seems fast on the mass storage node setup. Possibly a different mysql driver was used which is handling timeouts differently and did a more appropriate asynchronous check on the storage nodes. I am still digging through the code. Fairly sure I can come up with a fix in the next couple of days.

[Sebastian Roth]

@wanderson Would you please test the latest dev-branch version? You should see an improvement if only upgrading the master node already. But to get to experience the full scope of changes I made you’d need to upgrade all storage nodes as well!

[wanderson]

@Sebastian-Roth

Okay, I’ll test it, and then put the result here.

[wanderson]

@Sebastian-Roth said in FOG 1.5.5 SLOW OPEN STORAGES NODES:

@wanderson Would you please test the latest dev-branch version? You should see an improvement if only upgrading the master node already. But to get to experience the full scope of changes I made you’d need to upgrade all storage nodes as well!

I tried to install via git, but I could not, where I get the .zip of the branch version??

[Sebastian Roth]

@wanderson said in FOG 1.5.5 SLOW OPEN STORAGES NODES:

I tried to install via git, but I could not, where I get the .zip of the branch version??

Errors you got when trying git?

git clone https://github.com/FOGProject/fogproject
cd fogproject/
git checkout dev-branch
cd bin/
./installfog.sh

But sure you can download the ZIP archive from github as well: https://github.com/FOGProject/fogproject/archive/dev-branch.zip

[wanderson]

@Sebastian-Roth said in FOG 1.5.5 SLOW OPEN STORAGES NODES:

@wanderson Would you please test the latest dev-branch version? You should see an improvement if only upgrading the master node already. But to get to experience the full scope of changes I made you’d need to upgrade all storage nodes as well!

Watch video new test: -------------> https://youtu.be/r4T6oDEhQhQ

[Tom Elliott]

@wanderson That’s much faster than in the past. This is because we’re also testing if the machines are actually available and turned on. We do want to know if a machine is online and accessible right?

[wanderson]

@Tom-Elliott said in FOG 1.5.5 SLOW OPEN STORAGES NODES:

@wanderson That’s much faster than in the past. This is because we’re also testing if the machines are actually available and turned on. We do want to know if a machine is online and accessible right?

would it be this ------> https://youtu.be/I5Mr8dJIMpo

[Sebastian Roth]

@wanderson Compared to your first video it improved immensely as you couldn’t even load the pages with 1.5.4 and now we improved to only 5 second delay when loading the storage node list or configuration page - for checking 35 storage nodes to be online!!

As I already said, if you update all your storage nodes to dev-branch it would be even faster!

@wanderson said:

would it be this ------> https://youtu.be/I5Mr8dJIMpo

Yes, one of your storage nodes seems a bit slower than the others. Because we test the online status on nodes in FOG 1.5.x it takes a little bit of time to load.

I would suggest you update the slowest storage nodes to dev-branch as well (best if you upgrade all of them!) and see if that helps.

Sure we could improve page loading speed even more if we do the online status check using AJAX.