Web GUI error after clicking List Hosts

[tesparza]

@george1421 I copy pasted the command but its not giving the results. Im running CentOS and MariaDB

[george1421]

@tesparza Ok then you have an older version of mariadb so we can just keep moving forward with the migration.

I’m referencing this post: https://forums.fogproject.org/post/133490

At this point we need to temporarily stop apache so we don’t have any transactions going on when you run the migration command. Do this with: systemctl stop httpd on centos.

The next steps are to run this command via the mysql cli program:

SELECT TABLE_NAME,ENGINE
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = 'fog' and ENGINE = 'MyISAM';

The output should show that all tables are in MyISAM format.

Now paste this section into the cli. You might want to do it a few lines at a time in case there is an error message.

USE FOG
ALTER TABLE LDAPServers ENGINE=InnoDB;
ALTER TABLE clientUpdates ENGINE=InnoDB;
ALTER TABLE dirCleaner ENGINE=InnoDB;
ALTER TABLE globalSettings ENGINE=InnoDB;
ALTER TABLE greenFog ENGINE=InnoDB;
ALTER TABLE groupMembers ENGINE=InnoDB;
ALTER TABLE groups ENGINE=InnoDB;
ALTER TABLE history ENGINE=InnoDB;
ALTER TABLE hookEvents ENGINE=InnoDB;
ALTER TABLE hostAutoLogOut ENGINE=InnoDB;
ALTER TABLE hostMAC ENGINE=InnoDB;
ALTER TABLE hostScreenSettings ENGINE=InnoDB;
ALTER TABLE hosts ENGINE=InnoDB;
ALTER TABLE imageGroupAssoc ENGINE=InnoDB;
ALTER TABLE imagePartitionTypes ENGINE=InnoDB;
ALTER TABLE imageTypes ENGINE=InnoDB;
ALTER TABLE images ENGINE=InnoDB;
ALTER TABLE imagingLog ENGINE=InnoDB;
ALTER TABLE inventory ENGINE=InnoDB;
ALTER TABLE ipxeTable ENGINE=InnoDB;
ALTER TABLE keySequence ENGINE=InnoDB;
ALTER TABLE location ENGINE=InnoDB;
ALTER TABLE locationAssoc ENGINE=InnoDB;
ALTER TABLE moduleStatusByHost ENGINE=InnoDB;
ALTER TABLE modules ENGINE=InnoDB;
ALTER TABLE multicastSessions ENGINE=InnoDB;
ALTER TABLE multicastSessionsAssoc ENGINE=InnoDB;
ALTER TABLE nfsFailures ENGINE=InnoDB;
ALTER TABLE nfsGroupMembers ENGINE=InnoDB;
ALTER TABLE nfsGroups ENGINE=InnoDB;
ALTER TABLE notifyEvents ENGINE=InnoDB;
ALTER TABLE os ENGINE=InnoDB;
ALTER TABLE oui ENGINE=InnoDB;
ALTER TABLE plugins ENGINE=InnoDB;
ALTER TABLE powerManagement ENGINE=InnoDB;
ALTER TABLE printerAssoc ENGINE=InnoDB;
ALTER TABLE printers ENGINE=InnoDB;
ALTER TABLE pxeMenu ENGINE=InnoDB;
ALTER TABLE scheduledTasks ENGINE=InnoDB;
ALTER TABLE schemaVersion ENGINE=InnoDB;
ALTER TABLE snapinAssoc ENGINE=InnoDB;
ALTER TABLE snapinGroupAssoc ENGINE=InnoDB;
ALTER TABLE snapinJobs ENGINE=InnoDB;
ALTER TABLE snapinTasks ENGINE=InnoDB;
ALTER TABLE snapins ENGINE=InnoDB;
ALTER TABLE supportedOS ENGINE=InnoDB;
ALTER TABLE taskLog ENGINE=InnoDB;
ALTER TABLE taskStates ENGINE=InnoDB;
ALTER TABLE taskTypes ENGINE=InnoDB;
ALTER TABLE tasks ENGINE=InnoDB;
ALTER TABLE userCleanup ENGINE=InnoDB;
ALTER TABLE userTracking ENGINE=InnoDB;
ALTER TABLE users ENGINE=InnoDB;
ALTER TABLE virus ENGINE=InnoDB;
ALTER TABLE wolbroadcast ENGINE=InnoDB;

And the end rerun this command

SELECT TABLE_NAME,ENGINE
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = 'fog';

You should now see all of the tables in innodb format. If you get an error message around an invalid date format then we will need to dig a bit deeper. I think on the debian based systems there was 3 possible tables. When I first did this on centos it worked correctly.

Now that all of the tables have been updated, exit the mysql cli and restart apache. systemctl start httpd

That should be it. Monitor top for about 10-20 minutes. You should not see 70% cpu on mysql again. Monitor for the next few hours and everything stay normal.

[tesparza]

@george1421 Okay I did the commands, some did error out saying they dont exist but this are the results of the final command. Website seems to be running smoother. Will continue to monitor.

+------------------------+--------+
| TABLE_NAME             | ENGINE |
+------------------------+--------+
| clientUpdates          | InnoDB |
| dirCleaner             | InnoDB |
| fileChecksums          | MyISAM |
| globalSettings         | InnoDB |
| greenFog               | InnoDB |
| groupMembers           | InnoDB |
| groups                 | InnoDB |
| history                | InnoDB |
| hookEvents             | InnoDB |
| hostAutoLogOut         | InnoDB |
| hostMAC                | InnoDB |
| hostScreenSettings     | InnoDB |
| hosts                  | InnoDB |
| hoststatus             | MyISAM |
| imageGroupAssoc        | InnoDB |
| imagePartitionTypes    | InnoDB |
| imageTypes             | InnoDB |
| images                 | InnoDB |
| imagingLog             | InnoDB |
| inventory              | InnoDB |
| ipxeTable              | InnoDB |
| keySequence            | InnoDB |
| location               | InnoDB |
| locationAssoc          | InnoDB |
| moduleStatusByHost     | InnoDB |
| modules                | InnoDB |
| multicastSessions      | InnoDB |
| multicastSessionsAssoc | InnoDB |
| nfsFailures            | InnoDB |
| nfsGroupMembers        | InnoDB |
| nfsGroups              | InnoDB |
| notifyEvents           | InnoDB |
| os                     | InnoDB |
| oui                    | InnoDB |
| plugins                | InnoDB |
| powerManagement        | InnoDB |
| printerAssoc           | InnoDB |
| printers               | InnoDB |
| pxeMenu                | InnoDB |
| scheduledTasks         | InnoDB |
| schemaVersion          | InnoDB |
| snapinAssoc            | InnoDB |
| snapinGroupAssoc       | InnoDB |
| snapinJobs             | InnoDB |
| snapinTasks            | InnoDB |
| snapins                | InnoDB |
| supportedOS            | InnoDB |
| taskLog                | InnoDB |
| taskStates             | InnoDB |
| taskTypes              | InnoDB |
| tasks                  | InnoDB |
| userCleanup            | InnoDB |
| userTracking           | InnoDB |
| users                  | InnoDB |
| virus                  | InnoDB |
+------------------------+--------+
55 rows in set (0.00 sec)

[george1421]

@tesparza OK I see two tables either I missed or there was an error converting.

Can you run these commands

use fog
ALTER TABLE fileChecksums ENGINE=InnoDB;
ALTER TABLE hoststatus ENGINE=InnoDB;

You can do this now or later. You got the big tables that cause resource contention already, these are just for completeness.

I’m glad the innodb switch has settled down your FOG server, lets see how it runs for the next day or so to see if we need to tune it a bit more.

[tesparza]

@george1421 Okay now everything is InnoDB, Server is running bad, web gui taking long time to load, can’t image. CPU usage is around 40%

FYI this is running of a Optiplex 7010 i5 quad core, 8gb ram

Do I maybe need to switch to a server?

[george1421]

@tesparza Ok the innodb engine is much more efficient than myisam, so the effort is not wasted. It will never be worse with innodb format.

So what is your top process now? Post a screen shot of top sorted by Process.

How many computers with FOG clients?
What is your client check in interval? Still the default of 5 seconds?

Don’t change anything lets see where we are at first.

[tesparza]

@george1421

2000 + clients

Interval checkin time it 60 seconds

[george1421]

@tesparza For testing lets change the check in interval to 300 seconds (5 minutes) and then lets wait 15 minutes to see if things settle down. This isn’t the answer at the moment, its just to see if having 2000 hosts hit (if we lineralize it, 34 per second) is a bit too much for your hardware.

The screen shot you shown is indicative of what a myisam engine configuration will give us. We know we already fixed that.

Looking at top, it also appears you only have 4GB of ram. Can you do a uname -a and post the results here?

[tesparza]

@george1421 Okay interval changed to 300

[george1421]

@tesparza well that’s a good thing you have a 64 bit OS. Since you are telling me 8GB is installed but top is reporting only 4GB then I was thinking a 32 bit OS.

If the 300 second check in doesn’t settle things down, then please confirm you have 8GB installed. Since is a 7010 you might be able to find some old memory to top it out at 16GB. I know I have at least 10 sticks of 4GB 7010 memory in the bottom of my junk bin. I’m hoping you have similar.

Again when things settle down a bit we’ll check also disk performance. Right now the sql sever is having a high cpu load. This could be either disk, or not enough free ram for caching to happen. We need to make small changes to see where things pick up the most.

[tesparza]

@george1421 Okay it seems to be running smoother, I will add more RAM, eventually I will transfer to a server.

[george1421]

@tesparza I made a post a few years ago to test different hardware and its impact on imaging speed. We may need to reference it as we work through this.

https://forums.fogproject.org/topic/10459/can-you-make-fog-imaging-go-fast

If your fog server has a sata spinning hard drive that may also be the culprit. But again testing will show us.

[tesparza]

@george1421 Okay I have upgraded the RAM to 16gb. Will see how it goes. Thank you.

[george1421]

@tesparza Did you have 4GB in there when you opened the machine? Does top now show 16GB of ram? I’m kind of interested in why things were reporting as they were.

[tesparza]

@george1421

Running smooth so far. Yes it had 4GB before.

[george1421]

@tesparza Its good we can point our finger at low memory as the problem. The small cache space on 4GB is only left 800MB of room so mysql had to commit everything to disk instead of caching it for later. (speculation) I did find it surprising that in the previous top screen shot, you had 4GB of swap space but none was used even when the system was low on ram. It wouldn’t have really helped in your case, but I find it strange that no swap space was used.

Now that you found an acceptable level you can start decreasing your fog client check in time till you find a happy balance between speed and frequency. The only issue I can think of by keeping the check in time to 5 minutes would be during post imaging if the fog client was used to rename the system, connect to AD, or do a scheduled reboot. Its possible the client computer wouldn’t react to the command for about 5 minutes.

[tesparza]

This post is deleted!