Fog web interface times out loading almost any page


  • Testers

    Ubuntu 14 SVN 4602 web pages fail to load at all unless retried several times this is no matter which icon is clicked the apache error log shows this error repeating.

    [Fri Sep 11 13:08:44.997022 2015] [:error] [pid 9764] [client 10.109.49.75:51920] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 64
    [Fri Sep 11 13:08:44.997054 2015] [:error] [pid 9764] [client 10.109.49.75:51920] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 64
    [Fri Sep 11 13:08:44.997101 2015] [:error] [pid 9764] [client 10.109.49.75:51920] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 64
    [Fri Sep 11 13:08:44.997135 2015] [:error] [pid 9764] [client 10.109.49.75:51920] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 64
    [Fri Sep 11 13:08:44.997167 2015] [:error] [pid 9764] [client 10.109.49.75:51920] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 64
    [Fri Sep 11 13:08:44.997199 2015] [:error] [pid 9764] [client 10.109.49.75:51920] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 64
    [Fri Sep 11 13:08:44.997230 2015] [:error] [pid 9764] [client 10.109.49.75:51920] PHP Warning: mysqli::query(): Couldn’t fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 64
    [Fri Sep 11 13:08:44.997262 2015] [:error] [pid 9764] [client 10.109.49.75


  • Testers

    Ok bit the bullet and restored a backup of the database seems to have fixed the issue I will keep an eye on how it behaves under load today.


  • Testers

    So the main fog page shows active and pending tasks but going to task management loads a blank page and truncate jobs and snapins affects 0 rows any ideas this seems to be the only show stopper issue I still have.


  • Moderator

    @Joseph-Hales said:

    If I import an older backup made from the web page will it only add records that no longer exist or will it overwrite and delete the new records added since the backup?

    An import will blast your DB and place the import in place. You will loose all changes made since that backup was made.
    Of course this doesn’t affect the ACTUAL images made since then, it just affects the image definitions in the DB. Those can be recreated manually via the Web UI. But you have to create them exactly as they were (image type, OS, Exact path and the path is case sensitive).

    And I’ll make note in the WiKi about Ubuntu 14.


  • Testers

    Ok pages seem better and server load is down but my Task Management page fails to load at all, every other menu option loads normally.


  • Testers

    Ok if someone could make a note in the wiki the my.cnf in ubuntu 14 seems to be located in /etc/mysql/ not the root of /etc/ that fixed my too many connections. It seems the update to 14 deleted my old my.cnf and reverted to the default 100 max connections. after clearing the bad records it seems I lost 1050 records. If I import an older backup made from the web page will it only add records that no longer exist or will it overwrite and delete the new records added since the backup?


  • Moderator

    @Joseph-Hales said:

    I thought that command just saved the fog settings not the hosts and images?

    That feature exports the entire DB. Everything.


  • Moderator

    @Joseph-Hales said:

    Still run into the error ERROR 1040 (HY000): Too many connections

    I know what your issue is… you have tons of hosts… you need to increase your maximum connections!!! :-D

    There’s a section in this about just that :-) https://wiki.fogproject.org/wiki/index.php/Troubleshoot_MySQL


  • Testers

    Still run into the error ERROR 1040 (HY000): Too many connections


  • Testers

    When I try to connect to mysql it errors out with error too many connections I will update svn, reboot and try again.


  • Testers

    I thought that command just saved the fog settings not the hosts and images?


  • Moderator

    @Joseph-Hales said:

    No its not virtualized and if I loose some client records its not the end of the world I may be seeing this issues as well if the mac address’s are deleted should I clean out the database of these records. If so can you give me the mysql command to do this

    I’m 100% sure we can give you a command to do that, but if you have a host export you can just re import the data.

    I have not tested this so be forewarned… backup your DB before hand.
    FOG Configuration -> Configuration Save -> Export

    DELETE FROM hosts WHERE hostID NOT IN (SELECT hmhostID FROM hostMAC);
    

  • Testers

    No its not virtualized and if I loose some client records its not the end of the world I may be seeing this issues as well if the mac address’s are deleted should I clean out the database of these records. If so can you give me the mysql command to do this


  • Moderator

    @Joseph-Hales said:

    I also am only running the legacy client at this time. Finally the home page is always displaying that there are pending MAC’s that need approval but when I go to approve it fails to find any.

    Is your setup virtualized and do you have a snapshot ? Do you have DB backups available ?

    For example, I take a snapshot before every update, and export my DB stuff before every update. I keep the last two snapshots and ALL the DB backups. Plus I use samba to easily backup my images too.

    I only ask about the backups because of this :-\ https://forums.fogproject.org/topic/5716/some-clients-disappear-from-web-interface-but-are-still-present-in-databse


  • Testers

    I also am only running the legacy client at this time. Finally the home page is always displaying that there are pending MAC’s that need approval but when I go to approve it fails to find any.


  • Testers

    FOG_SERVICE_CHECKIN_TIME 60
    Hosts found 6781
    I updated to SVN 4666 this morning I don’t see the failure to load issue so far but not many users are logged in yet but page load times are about 5 seconds on the host search page when searching for one host and listing all hosts takes about 50 seconds I also get the install database upgrade page occasionally at random times.


  • Moderator

    @Joseph-Hales That’s a lot of Apache2 there, Joe…

    How many clients do you have running the fog client? what’s your check-in time set to?


  • Testers

    Here is what top looks like.
    top - 10:45:03 up 3 days, 2:16, 1 user, load average: 10.50, 8.93, 7.66
    Tasks: 470 total, 3 running, 467 sleeping, 0 stopped, 0 zombie
    %Cpu(s): 21.1 us, 2.6 sy, 0.0 ni, 74.2 id, 1.5 wa, 0.0 hi, 0.6 si, 0.0 st
    KiB Mem: 24678932 total, 24463228 used, 215704 free, 998420 buffers
    KiB Swap: 4844540 total, 0 used, 4844540 free. 20614724 cached Mem

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    13693 mysql 20 0 8722412 351992 7352 S 163.1 1.4 4024:15 mysqld
    7375 www-data 20 0 564848 255004 7280 R 89.2 1.0 0:55.44 apache2
    7659 www-data 20 0 327004 16804 5728 S 6.0 0.1 0:32.13 apache2
    1617 www-data 20 0 326748 17032 6084 S 5.3 0.1 1:22.77 apache2
    7637 www-data 20 0 326524 16484 5732 S 3.0 0.1 0:31.86 apache2
    5722 www-data 20 0 327076 17464 6112 S 2.7 0.1 0:54.97 apache2
    1659 www-data 20 0 327036 19084 7796 S 2.3 0.1 1:38.26 apache2
    5847 www-data 20 0 327004 17320 6028 S 2.3 0.1 0:53.62 apache2
    7322 www-data 20 0 327036 17148 5988 S 2.3 0.1 0:36.12 apache2
    7445 www-data 20 0 326748 17128 6048 S 2.3 0.1 0:38.99 apache2
    7451 www-data 20 0 327004 17220 6020 S 2.3 0.1 0:34.73 apache2
    7527 www-data 20 0 326748 16972 6008 S 2.3 0.1 0:33.10 apache2
    7550 www-data 20 0 327168 17532 6000 S 2.3 0.1 0:34.11 apache2
    7607 www-data 20 0 326972 17036 5984 S 2.3 0.1 0:32.40 apache2
    7611 www-data 20 0 326748 16988 5976 S 2.3 0.1 0:34.47 apache2
    7711 www-data 20 0 476540 20000 8728 S 2.3 0.1 0:32.78 apache2
    7713 www-data 20 0 326972 17152 5988 S 2.3 0.1 0:32.41 apache2
    1262 www-data 20 0 327004 17472 6144 S 2.0 0.1 1:34.06 apache2
    1691 www-data 20 0 326672 18168 7260 S 2.0 0.1 1:26.96 apache2
    1950 www-data 20 0 326748 17064 6080 S 2.0 0.1 1:22.28 apache2
    5724 www-data 20 0 326972 17364 6064 S 2.0 0.1 0:54.74 apache2
    5771 www-data 20 0 326492 16724 6036 S 2.0 0.1 0:54.01 apache2
    5851 www-data 20 0 327216 17512 6032 S 2.0 0.1 0:53.55 apache2
    7345 www-data 20 0 327004 17096 6004 S 2.0 0.1 0:35.27 apache2
    7374 www-data 20 0 326972 17148 6052 S 2.0 0.1 0:33.98 apache2
    7415 www-data 20 0 327004 17312 6008 S 2.0 0.1 0:35.58 apache2
    7431 www-data 20 0 326972 17104 5992 S 2.0 0.1 0:33.74 apache2
    7433 www-data 20 0 326972 17260 5992 S 2.0 0.1 0:36.23 apache2
    7492 www-data 20 0 326748 16852 5824 S 2.0 0.1 0:33.44 apache2
    7511 www-data 20 0 327004 17188 6008 S 2.0 0.1 0:32.97 apache2
    7524 www-data 20 0 326716 16972 5996 S 2.0 0.1 0:33.87 apache2
    7564 www-data 20 0 326460 16776 5984 S 2.0 0.1 0:33.87 apache2
    7576 www-data 20 0 326716 16864 5980 S 2.0 0.1 0:32.53 apache2
    7582 www-data 20 0 326492 16832 6008 S 2.0 0.1 0:33.65 apache2
    7592 www-data 20 0 327004 18208 7124 S 2.0 0.1 0:31.81 apache2
    7617 www-data 20 0 326972 17296 6008 S 2.0 0.1 0:33.30 apache2
    7649 www-data 20 0 326972 17296 5996 S 2.0 0.1 0:32.44 apache2


  • Moderator

    @Joseph-Hales What sort of processing power does your fog server have?

    Can you try to monitor the CPU usage via “top” ?


  • Testers

    it does seem to go away if my active users is low typically when the issues occurs i have 5 or more users logged in.


Log in to reply
 

357
Online

39.3k
Users

11.0k
Topics

104.4k
Posts

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.