Disabling several pieces of Javascript to improve performance on large databases

[andjjru]

Based upon the [URL=‘http://www.fogproject.org/wiki/index.php/Testimonials#Madison_Metropolitan_School_District.2C_Madison.2C_WI.2C_USA’]Testimonials page on the WIKI[/URL], our FOG deployment is about the largest out there, so I’m not sure how many others have run into the sluggishness on the web interface out of the box that we have. I just had to replicate a few changes we had done in the past after doing a total re-install on our server for Ubuntu 14.04 and FOG 1.0.1, so I figured this might be useful to others and belong somewhere other than a text file on my Dropbox.

Host Search - Disable search while typing (Also affects group and image searches):
[CODE]

/var/www/fog/management/js/fog.js (depending on which distro you choose)

#Replace
$this.keyup(function()
{
if (this.SearchTimer) clearTimeout(this.SearchTimer);

this.SearchTimer = setTimeout(function()
{
    PerformSearch();
            }, Options.SearchDelay);

#with
$this.keypress(function(e)
{
if(e.keyCode==13){
if (this.SearchTimer) clearTimeout(this.SearchTimer);

        this.SearchTimer = setTimeout(function()
        {
            PerformSearch();
        }, Options.SearchDelay);
        }

[/CODE]
Disable pinging of Host Search results:
[CODE]

/var/www/fog/management/js/fog.js (depending on which distro you choose)

#COMMENT OUT THIS LINE
$(‘.ping’, Container).fogPing();[/CODE]
Disable Dashboard bandwidth graphs:
[CODE]

/var/www/fog/management/js/fog.dashboard.js (depending on which distro you choose)

GraphBandwidth = $('#graph-bandwidth', '#content-inner');
GraphBandwidthFilterTransmit = $('#graph-bandwidth-filters-transmit', '#graph-bandwidth-filters');
GraphBandwidthFilterTransmitActive = GraphBandwidthFilterTransmit.hasClass('active');

// Bandwidth Graph - init plot
GraphBandwidthPlot = $.plot(GraphBandwidth, [[0,0]],
{
    'colors': ['#7386AD', '#91a73c'],
    'xaxis':
    {
        'mode':        'time'
    },
    'yaxis':
    {
        'min':        '0',
        'tickFormatter': function (v)
        {
            return v + ' MB/s';
        }
    },
    'series':
    {
        'lines':
        {
            'show': true
        }
    },
    'legend':
    {
        show:        false,
    }
});

// Bandwidth Graph - TX/RX Filter
$('#graph-bandwidth-filters-transmit, #graph-bandwidth-filters-receive', '#graph-bandwidth-filters').click(function()
{
    // Blur -> add active class -> remove active class from old active item
    $(this).blur().addClass('active').siblings('a').removeClass('active');
 
    // Update title
    $('#graph-bandwidth-title > span').eq(0).html($(this).html());
 
    // Set variable
    GraphBandwidthFilterTransmitActive = (GraphBandwidthFilterTransmit.hasClass('active') ? true : false)
 
    // Update graph
    UpdateBandwidthGraph();
 
    // Prevent default action
    return false;
});

// Bandwidth Graph - Time Filter
$('#graph-bandwidth-filters div:eq(2) a').click(function()
{
    // Blur -> add active class -> remove active class from old active item
    $(this).blur().addClass('active').siblings('a').removeClass('active');

    // Update title
    $('#graph-bandwidth-title > span').eq(1).html($(this).html());

    // Update max data points variable
    GraphBandwidthMaxDataPoints = $(this).attr('rel');

    // Update graph
    UpdateBandwidthGraph();

    // Prevent default action
    return false;
});

// Bandwidth Graph - start thread
setTimeout(function()
{
    UpdateBandwidth();
}, (200));[/CODE]

Before applying these changes, the web interface was generally slow and unstable and mysql and/or apache would be pegging at least one CPU core and using a lot of memory. After making the above changes, memory usage is about ~1GB or under, CPU usage is fairly light, and the web interface is fast and completely usable.

[Tom Elliott]

@andjjru

I’ve added these files to the svn trunk. I’m not using them directly, but a simple mv the other files to backup names and renaming of these files as their respective counterparts will work.

[andjjru]

Sweet!

I did still have some other slowness after this when modifying larger groups, I’ve been tweaking MySQL settings in my.cnf to alleviate that. Here’s my most recent version, this would vary somewhat depending on the beefiness of your server’s hardware.

[CODE]#

The MySQL database server configuration file.

You can copy this to one of:

- “/etc/mysql/my.cnf” to set global options,

- “~/.my.cnf” to set user-specific options.

One can use all long options that the program supports.

Run program with --help to get a list of available options and with

–print-defaults to see which it would actually understand and use.

For explanations see

http://dev.mysql.com/doc/mysql/en/server-system-variables.html

This will be passed to all mysql clients

It has been reported that passwords should be enclosed with ticks/quotes

escpecially if they contain “#” chars…

Remember to edit /etc/mysql/debian.cnf when changing the socket location.

[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock

Here is entries for some specific programs

The following values assume you have at least 32M ram

This was formally known as [safe_mysqld]. Both versions are currently parsed.

[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0

[mysqld]

* Basic Settings

user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking

Instead of skip-networking the default is now to listen only on

localhost which is more compatible and is not less secure.

bind-address = 127.0.0.1

* Fine Tuning

tmp_table_size = 128M
max_heap_table_size = 128M
key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8

This replaces the startup script and checks MyISAM tables if needed

the first time they are touched

myisam-recover = BACKUP
max_connections = 500
#table_cache = 64
thread_concurrency = 16

* Query Cache Configuration

query_cache_limit = 1M
query_cache_size = 64M

* Logging and Replication

Both location gets rotated by the cronjob.

Be aware that this log type is a performance killer.

As of 5.1 you can enable the log at runtime!

#general_log_file = /var/log/mysql/mysql.log
#general_log = 1

Error log - should be very few entries.

log_error = /var/log/mysql/error.log

Here you can see queries with especially long duration

#log_slow_queries = /var/log/mysql/mysql-slow.log
#long_query_time = 2
#log-queries-not-using-indexes

The following can be used as easy to replay backup logs or for replication.

note: if you are setting up a replication slave, see README.Debian about

other settings you may need to change.

#server-id = 1
#log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
slow-query-log = 1
slow-query-log-file = /var/log/mysql/mysql-slow.log
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name

* InnoDB

InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.

Read the manual for more InnoDB related options. There are many!

* Security Features

Read the manual, too, if you want chroot!

chroot = /var/lib/mysql/

For generating SSL certificates I recommend the OpenSSL GUI “tinyca”.

ssl-ca=/etc/mysql/cacert.pem

ssl-cert=/etc/mysql/server-cert.pem

ssl-key=/etc/mysql/server-key.pem

[mysqldump]
quick
quote-names
max_allowed_packet = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer = 512M

* IMPORTANT: Additional settings that can override those from this file!

The files must end with ‘.cnf’, otherwise they’ll be ignored.

!includedir /etc/mysql/conf.d/[/CODE]

I imagine some of my performance issues have to do with choosing to install on Ubuntu 14.04? I’ve seen the talk of it being a problem, but haven’t seen anyone list anything specific.

[Tom Elliott]

Some more help in speeding up your interface could be to disabling the animation effects in:

FOG Configuration->FOG Settings->General Settings->FOG_USE_ANIMATION_EFFECTS Uncheck the box and save.

Enjoy the speed increase.

[Mark Buteyn]

Somewhat old thread, but you can disable the ping on the host lists by going to FOG Configuration->FOG Settings->General Settings->FOG_HOST_LOOKUP and clearing the checkbox. Easier than disabling the javascript file. There’s a possibility this was introduced in the time between your post and my find, but if not, it’s here for others.