FOG Web GUI speed and default storage activity
-
@jgallo said in FOG Web GUI speed and default storage activity:
could I raise the number of vCPU on Hyper-V from 2 to 4? Will this help improve performance?
That would only help if your host system isn’t overburdened. If you have too many VMs on it already with too many cores assigned, and not enough cores available, it’ll just make things worse. But if you have plenty of resources, then it would help.
Also, set your client checkin time to something like 300 seconds (five minutes) and see if that makes a difference. Keep in mind the change here isn’t immediate - the clients have to checkin once more to actually get the new setting.
-
I don’t have too many VM’s. I do have 4 but each of those has either 1 or 2 vCPU’s allocated to them. I have now set 4 vCPU’s for the fog server VM and I still have that issue. I also set the client checkin time to 300 with same issue.
Here is what I noticed. I see that the FOG server disk I/O is about 15% give or take on a constant basis. I also noticed that all the disk activity is from apache2 with user www-data and mysql is using up to 5GB of memory at times just on idle. Could this be some programming bug or my database needs to be cleared?
-
@jgallo How many hosts do you have in your environment that have the FOG Client installed?
-
I don’t have any. We use Group Policy to manage printers, settings, etc. Back in the day we used the FOG Client when autojoin domain features were utilized. We don’t anymore due to large amounts of chromebooks replacing aging PC’s.
-
@jgallo What are the link speeds between the main fog server and the other nodes? How many images do you have? What’s the FOG IMAGE REP SLEEP TIME in fog settings set to?
-
At the secondary schools, the connection speed to our district office is 1Gb and the primary schools are at 100mb. The FOG IMAGE REP SLEEP TIME is set to 10800.
-
@jgallo How consistent is the problem? The “a valid connection cannot be established” problem. Any rhyme or pattern? Is this when imaging is happening?
-
on working branch 57 it was very consistent even with all the changes made to fog and the vCPU’s. I have been updating all the storage nodes and fog to working branch 64 today and the problem is still persistent. The only pattern I have observed is that upon rebooting the fog sever, the valid connection messages do not appear for about an hour or so. Then the messages begin to appear for random nodes that I have the graph enabled. At random times, the messages tend to go away but then come back upon selecting another storage node on the dashboard.
-
@jgallo Do you know what version of fog this problem started with?
-
I know that I went from 1.4.4 to 1.5.0 RC1 if I recalled. I know when I upgraded I made a huge leap. New interface and all. During that time there were replication issues and eventually updated to 1.5.0 RC7 which still had replication issues. I then upgraded to 1.5.0 RC9 which replication had major issue that was resolved in a working branch. So I have been on working branch 57 until about an hour ago I went to 64 with all nodes and fog server. I have been observing this issue since I have been on 1.5.0 from the new interface. I never had this issue in 1.4.4
-
@jgallo said in FOG Web GUI speed and default storage activity:
The only pattern I have observed is that upon rebooting the fog sever, the valid connection messages do not appear for about an hour or so.
I’m thinking about this - I’d like you to try to restart only Apache and see if it has the same effect or not. On CentOS/Fedora/RHEL it’s
systemctl restart httpdand on Ubuntu 16/Debian8/Debian9 it’ssystemctl restart apache2and on Ubuntu 14-,debian7- it’sservice apache2 restart -
Tried that and still get the database connection message.
-
@jgallo During when the problem is happening, what does this command return?
free -h;uptime -
@wayne-workman
I have rebooted due to upgrade to working branch. FYI.total used free shared buff/cache available Mem: 15G 510M 181M 47M 14G 14G Swap: 4.0G 268K 4.0G 15:59:54 up 1:56, 2 users, load average: 0.24, 0.19, 0.21``` -
@jgallo You have 14GB cached in RAM, which I’ll say is substantial. You can clear that with the below command:
sudo sh -c "sync; echo 3 > /proc/sys/vm/drop_caches"
Does the issue resolve after this? Also, checkfree -hafterwards. -
went down to 139MB. LOL dam that was a lot of cache. Wish I had it in my pocket.
-
This was just a minute of running free -h
total used free shared buff/cache available Mem: 15G 427M 14G 47M 289M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 434M 14G 47M 289M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 484M 14G 47M 292M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 485M 14G 47M 292M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 486M 14G 47M 301M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 486M 14G 47M 301M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 487M 14G 47M 301M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 487M 14G 47M 301M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 488M 14G 47M 301M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 488M 14G 47M 302M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 490M 14G 47M 302M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 489M 14G 47M 302M 14G Swap: 4.0G 0B 4.0G administrator@VUSD-FOG:~$ free -h total used free shared buff/cache available Mem: 15G 491M 14G 47M 302M 14G``` -
@jgallo Most importantly, are the nodes reporting properly?
-
I rebooted server just in case. Went home for the day. I will keep an eye on the nodes and see how they are in the morning. Also see how fast the cache grows. It has gone up to 305MB right now but the web UI is substantially snapier.
-
@jgallo If this solves the problem, we can setup a cron job that does this every hour.
