MYSQL/HTTPD resource issues in 5020
-
If you know FOG Version 4960 was a “good point” then maybe you could use SVN revisions: 4159 thru 4189 (4159 = 4960 – git commit svn 3510 + number of commits since that tag was released) and 4189 = 5020 ) (this means 30 svn revisions have been pushed.)
You could go thru them going back and forth cutting the last by half.
For example, if 4159 is known good, and 4189 is known bad, split the difference in the middle. This means test svn version 4174. If this is good, then we know 4159 thru 4174 is good. If 4174 is bad, we know the problem came in before this point.
In either case you split it by half again from the last good/bad state.
For example, if 4174 is bad, we go back 7.5 revisions (just make it an even 7). If 4174 is good go up 7.5 revisions (just make it an even 8.) it will be much faster to determine exactly when the problem occurred.
Hopefully this makes sense. Don’t think of things in the sense of the versions. They basically get incremented by 2 each commit because of how we do the syncing of svn to git. We have to get the updates and push them, then merge the svn to the dev branch. So the svn branch should match exactly with svn IRL.
If you have to do the splits, going backward use floor to get the value (round down to the nearest integer). Going forward, use ceiling to get the value (round up to the nearest integer).
This way, you are only having to go thru maybe 6 - 7 revisions to know where the bad revision was introduced, rather than going thru 30 revisions one by one hoping to find it.
Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.
Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)
Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG
-
Apologies.
The @Developers are working on it. FOG has recently moved to innodb, and with that comes many challenges and necessary tweaks.
SVN version 4103 is pretty solid. But keep in mind FOG does not support downgrading.
Please provide any relevant logs. How many hosts do you have in your environment?
-
Tom is suggesting you figure out where it went wrong… somewhere between 4960 and 5020 obviously.
You can checkout a specific revision like this:
svn -r xxxx upor
svn -r 4174 upOf course, install it afterwards:
./installfog.sh -yIt really helps a lot if FOG Trunk is virtualized… because again I’d stress fog does not support downgrading… but if Tom says it’s OK to do for this small range of SVN revisions, then it’s OK to do.
-
I understand what you are saying and i am happy to help try and figure out where it went south. My only question is the DB schema backtracking…
Do you know how far i can go back in revisions before i hit a change that will cause a problem? I know fog can’t downgrade so i want to make sure i don’t blow up the DB
Adam
-
While it is never a good idea to revert, none of the recent db changes are going to affect anything. If anything else, they will simply be ignored in revisions where they were not in use. Where you would run into problems is if you try to backtrack say all the way back to version 1.2.0 on your current database. This is because I removed necessary bits that 1.2 expects, but is not necessary in the current. Just don’t downgrade directly to one such revision and you will be fine.
Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.
Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)
Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG
-
@Adam-Taylor You can export the DB before each change.
FOG Configuration -> Configuration Save -> Export
That exports the entire DB. Everything. I notate my exports name with a revision number post-download.
So all my exports are named “filename_rxxxx_month_year”
-
I forgot to add.
We have 1892 hosts currently.
As for the logs…the only logs i am getting are:
[Tue Oct 20 16:09:34 2015] [error] [client 10.69.15.60] PHP Warning: mysqli::real_connect(): (08004/1040): Too many connections in /var/www/html/fog/lib/db/MySQL.class.php on line 29
This is when the client hits the server to see if it has any tasks.
If i set the max_connections in sql to something huge (2k plus), in about 45 minutes i will start to get that message. The connections just keep racking up and up…never go down…
I also tried setting “wait_timeout” and “interactive_timeout” to something like 10 seconds to try and make the connections go down…nada…connections keep rising.
I’m about to get off work, but first thing in the morning i am going to start bouncing around the svn versions to see if i can find which one might be causing this.
Thanks!
Adam
-
@Tom-Elliott For me too, cpu is running 100% due to apache2 and mysqld processes after svn 5020 upgrade.
In web interface I’ve got these values : Load Average 96.24, 101.58, 106.29
Not the first time I’ve got this issue after upgrading svn
-
It’s seems ok with last svn (git 5040)
Load Average 1.16, 2.65, 2.48 -
@Matthieu-Jacquart et al, I believe this was fixed last night. Something didn’t seem right and while chatting what I thought the issue was it came to me. Thank you @Uncle-Frank and everybody for reporting and patience. Basically my suspicion of what was happening is the gui kept on make connections to the db. Anytime a new page was loaded or clients where checking in a new db connection was being created even if you were using persistent connections. I looked over the code and it certainly made sense based on the logic of the code. I should’ve thought of it sooner I know, but I’m glad it’s now working more appropriately.
-
Sorry, I haven’t gotten back lately…had a sick child.
I have tried 5040 and 5042 this morning and I am still seeing almost complete CPU usage to the point that the web gui will talk 20-30 seconds to respond or not respond at all or not render most of the page.
I will try to go back to older ones to see where the problem was introduced.
-
Actually I’d recommend doing a full server reboot. I think what you’re seeing now is it catching up from all the bad stuff before. Of course it may be the case that something else is weird. Can you watch your Apache error log? Often when I see delays of loading as such there is usually something at a fast and large pace writing to the log files.
-
Rebooted…1 hour later still seeing same thing. I’m thinking there is something going on with apache. While i write this there is 259 httpd processes chewing up everything.
In the apache log, i get:
[Thu Oct 22 16:54:22 2015] [error] server reached MaxClients setting, consider raising the MaxClients setting
I just don’t know what would be causing this all of a sudden… Was something changed in the apache v/host settings in the past few weeks with the installer?
Adam
-
And i forgot.
The message only happens every once and a while in the error_log file.
-
@Adam-Taylor No, nothing has been changed on the vhost generation. Maybe you have a lot more computers running and creating new connections?
-
@Adam-Taylor Do you have a lot of clients? Are you using FOG_HOST_LOOKUP? (On hosts do you see the green/red dots when on list/search?)
-
@Adam-Taylor Please install the packet
lsofon your FOG server and runlsof -i :80to see who keeps all those apache processes busy. -
Right now i have 33525 mysql connections when i run “show status like ‘Con%’;” in mysql cli. And no…that was not a mistype
The connections are just not going away.Total registered hosts are 1892 of which about 1800 have the fog client.
Running lsof, I am a around 20-30 established connections at any one given time with about the same listening. All while the CPU is around 100%. Every once and a while it might spike to around 180 or so but then drop back down within 30 seconds or so.
FOG_HOST_LOOKUP was checked and I just unchecked it to see if that helps. So far I am not seeing a difference after a total system reboot.
I had to restart the mysqld process so that I could get to that setting as the web gui would not do anything. In that 2 minutes from a reboot, I am back upto 297 mysql connections.
BTW, I am running this on RedHat Enterprise 6.
Thanks!
Adam
-
What does
lsof -i :3306show? Is it all localhost connections?And what about
lsof -i :80? All connections from different clients?? -
Hmm, i’m getting this now too! FOG server is being crippled even after a reboot and wiping the hosts/reimporting (and Host lookup isn’t checked)
I am assuming that its because we have lots of PCs checking in but I didn’t think that so many processes would get generated!
Also lsof for port 80:
root@fog:/var/log/mysql# lsof -i :80 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME apache2 2084 root 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 2636 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 2636 www-data 16u IPv6 205385 0t0 TCP fog.myserver.co.uk:http->10.1.24.175:49760 (ESTABLISHED) apache2 2768 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 2768 www-data 16u IPv6 211218 0t0 TCP fog.myserver.co.uk:http->10.28.10.238:49570 (ESTABLISHED) apache2 2807 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 2807 www-data 16u IPv6 206891 0t0 TCP fog.myserver.co.uk:http->10.9.110.159:57368 (ESTABLISHED) apache2 2818 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 2818 www-data 16u IPv6 208587 0t0 TCP fog.myserver.co.uk:http->10.1.26.98:49835 (ESTABLISHED) apache2 2863 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 2863 www-data 16u IPv6 210004 0t0 TCP fog.myserver.co.uk:http->10.28.10.167:53887 (ESTABLISHED) apache2 3093 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 3093 www-data 16u IPv6 211334 0t0 TCP fog.myserver.co.uk:http->10.1.24.122:65430 (ESTABLISHED) apache2 3097 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 3097 www-data 16u IPv6 211239 0t0 TCP fog.myserver.co.uk:http->10.1.21.170:29595 (ESTABLISHED) apache2 3579 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 3579 www-data 16u IPv6 208959 0t0 TCP fog.myserver.co.uk:http->10.9.110.108:56545 (ESTABLISHED) apache2 3617 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 3617 www-data 16u IPv6 208904 0t0 TCP fog.myserver.co.uk:http->10.9.112.158:64571 (ESTABLISHED) apache2 3657 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 3657 www-data 16u IPv6 207246 0t0 TCP fog.myserver.co.uk:http->10.9.110.166:58779 (ESTABLISHED) apache2 4124 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4124 www-data 16u IPv6 205230 0t0 TCP fog.myserver.co.uk:http->10.28.10.193:65050 (ESTABLISHED) apache2 4134 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4134 www-data 16u IPv6 207383 0t0 TCP fog.myserver.co.uk:http->10.240.11.209:60319 (ESTABLISHED) apache2 4160 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4160 www-data 16u IPv6 207332 0t0 TCP fog.myserver.co.uk:http->10.9.112.200:53346 (ESTABLISHED) apache2 4176 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4176 www-data 16u IPv6 204956 0t0 TCP fog.myserver.co.uk:http->10.28.10.21:49847 (ESTABLISHED) apache2 4190 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4190 www-data 16u IPv6 206864 0t0 TCP fog.myserver.co.uk:http->10.1.25.145:54472 (ESTABLISHED) apache2 4191 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4191 www-data 16u IPv6 208675 0t0 TCP fog.myserver.co.uk:http->10.1.26.148:60850 (ESTABLISHED) apache2 4205 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4205 www-data 16u IPv6 207418 0t0 TCP fog.myserver.co.uk:http->10.1.25.146:56839 (ESTABLISHED) apache2 4220 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4220 www-data 16u IPv6 208115 0t0 TCP fog.myserver.co.uk:http->pc24-6.myserver.co.uk:57599 (ESTABLISHED) apache2 4225 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4225 www-data 16u IPv6 209198 0t0 TCP fog.myserver.co.uk:http->10.9.106.35:49624 (ESTABLISHED) apache2 4669 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4669 www-data 16u IPv6 209862 0t0 TCP fog.myserver.co.uk:http->10.28.10.68:49459 (ESTABLISHED) apache2 4672 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4672 www-data 16u IPv6 209534 0t0 TCP fog.myserver.co.uk:http->10.240.11.198:49391 (ESTABLISHED) apache2 4675 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4675 www-data 16u IPv6 207291 0t0 TCP fog.myserver.co.uk:http->10.28.10.166:54133 (ESTABLISHED) apache2 4691 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4691 www-data 16u IPv6 207738 0t0 TCP fog.myserver.co.uk:http->10.28.10.21:49848 (ESTABLISHED) apache2 4694 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4694 www-data 16u IPv6 211263 0t0 TCP fog.myserver.co.uk:http->10.240.11.182:55943 (ESTABLISHED) apache2 4696 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4696 www-data 16u IPv6 209160 0t0 TCP fog.myserver.co.uk:http->10.1.23.237:56832 (ESTABLISHED) apache2 4735 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4735 www-data 16u IPv6 209401 0t0 TCP fog.myserver.co.uk:http->10.28.10.217:49597 (ESTABLISHED) apache2 4756 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4756 www-data 16u IPv6 209628 0t0 TCP fog.myserver.co.uk:http->10.240.11.162:64878 (ESTABLISHED) apache2 4768 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4768 www-data 16u IPv6 206502 0t0 TCP fog.myserver.co.uk:http->10.9.106.41:49645 (FIN_WAIT2) apache2 4777 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 4777 www-data 16u IPv6 209994 0t0 TCP fog.myserver.co.uk:http->10.9.106.26:50253 (ESTABLISHED) apache2 5100 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5100 www-data 16u IPv6 209727 0t0 TCP fog.myserver.co.uk:http->10.1.25.173:56715 (ESTABLISHED) apache2 5128 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5128 www-data 16u IPv6 207472 0t0 TCP fog.myserver.co.uk:http->10.28.10.213:53558 (ESTABLISHED) apache2 5520 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5520 www-data 16u IPv6 209087 0t0 TCP fog.myserver.co.uk:http->10.1.23.231:51926 (ESTABLISHED) apache2 5530 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5530 www-data 16u IPv6 211051 0t0 TCP fog.myserver.co.uk:http->10.240.11.162:64879 (ESTABLISHED) apache2 5533 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5533 www-data 16u IPv6 205433 0t0 TCP fog.myserver.co.uk:http->10.1.37.27:27514 (ESTABLISHED) apache2 5544 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5544 www-data 16u IPv6 205271 0t0 TCP fog.myserver.co.uk:http->10.1.23.148:49740 (ESTABLISHED) apache2 5551 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5551 www-data 16u IPv6 208381 0t0 TCP fog.myserver.co.uk:http->10.9.112.189:49842 (ESTABLISHED) apache2 5570 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5570 www-data 16u IPv6 211113 0t0 TCP fog.myserver.co.uk:http->10.9.106.40:49619 (ESTABLISHED) apache2 5572 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5572 www-data 16u IPv6 208036 0t0 TCP fog.myserver.co.uk:http->10.9.110.128:65140 (ESTABLISHED) apache2 5580 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5580 www-data 16u IPv6 206913 0t0 TCP fog.myserver.co.uk:http->10.1.23.230:49844 (ESTABLISHED) apache2 5602 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5602 www-data 16u IPv6 208805 0t0 TCP fog.myserver.co.uk:http->10.1.24.126:55469 (ESTABLISHED) apache2 5618 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5618 www-data 16u IPv6 206700 0t0 TCP fog.myserver.co.uk:http->10.9.106.29:49608 (ESTABLISHED) apache2 5619 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 5619 www-data 16u IPv6 208974 0t0 TCP fog.myserver.co.uk:http->10.1.55.236:49938 (ESTABLISHED) apache2 6006 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6006 www-data 16u IPv6 209326 0t0 TCP fog.myserver.co.uk:http->10.1.25.118:55158 (ESTABLISHED) apache2 6268 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6268 www-data 16u IPv6 209858 0t0 TCP fog.myserver.co.uk:http->10.1.26.123:53330 (ESTABLISHED) apache2 6269 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6269 www-data 16u IPv6 211026 0t0 TCP fog.myserver.co.uk:http->10.28.12.239:52159 (ESTABLISHED) apache2 6270 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6270 www-data 16u IPv6 211175 0t0 TCP fog.myserver.co.uk:http->10.28.10.93:64841 (ESTABLISHED) apache2 6273 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6273 www-data 16u IPv6 209592 0t0 TCP fog.myserver.co.uk:http->10.9.110.127:58944 (ESTABLISHED) apache2 6292 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6292 www-data 16u IPv6 207455 0t0 TCP fog.myserver.co.uk:http->10.1.21.151:51066 (ESTABLISHED) apache2 6296 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6296 www-data 16u IPv6 209338 0t0 TCP fog.myserver.co.uk:http->10.1.24.160:52020 (ESTABLISHED) apache2 6304 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6304 www-data 16u IPv6 211197 0t0 TCP fog.myserver.co.uk:http->10.9.112.168:54211 (ESTABLISHED) apache2 6305 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6305 www-data 16u IPv6 211145 0t0 TCP fog.myserver.co.uk:http->10.28.11.210:49781 (ESTABLISHED) apache2 6787 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6787 www-data 16u IPv6 209787 0t0 TCP fog.myserver.co.uk:http->pc26-36.myserver.co.uk:49781 (ESTABLISHED) apache2 6794 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6794 www-data 16u IPv6 208319 0t0 TCP fog.myserver.co.uk:http->10.1.23.233:49922 (ESTABLISHED) apache2 6796 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 6796 www-data 16u IPv6 199442 0t0 TCP fog.myserver.co.uk:http->10.240.11.176:50216 (ESTABLISHED) apache2 14670 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14670 www-data 16u IPv6 208028 0t0 TCP fog.myserver.co.uk:http->10.1.25.150:58961 (ESTABLISHED) apache2 14671 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14671 www-data 16u IPv6 209766 0t0 TCP fog.myserver.co.uk:http->10.28.10.168:60393 (ESTABLISHED) apache2 14672 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14672 www-data 16u IPv6 205388 0t0 TCP fog.myserver.co.uk:http->10.1.23.211:49845 (ESTABLISHED) apache2 14673 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14673 www-data 16u IPv6 211298 0t0 TCP fog.myserver.co.uk:http->10.240.11.236:55911 (ESTABLISHED) apache2 14681 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14681 www-data 16u IPv6 209099 0t0 TCP fog.myserver.co.uk:http->10.1.26.103:51299 (ESTABLISHED) apache2 14682 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14682 www-data 16u IPv6 205304 0t0 TCP fog.myserver.co.uk:http->10.9.112.196:49838 (ESTABLISHED) apache2 14684 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14684 www-data 16u IPv6 211173 0t0 TCP fog.myserver.co.uk:http->10.1.23.209:50288 (ESTABLISHED) apache2 14685 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14685 www-data 16u IPv6 209595 0t0 TCP fog.myserver.co.uk:http->10.1.24.103:49763 (ESTABLISHED) apache2 14686 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14686 www-data 16u IPv6 208799 0t0 TCP fog.myserver.co.uk:http->10.28.10.222:51282 (ESTABLISHED) apache2 14687 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14687 www-data 16u IPv6 208014 0t0 TCP fog.myserver.co.uk:http->10.1.26.194:49692 (ESTABLISHED) apache2 14694 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14694 www-data 16u IPv6 206769 0t0 TCP fog.myserver.co.uk:http->10.1.23.205:51748 (ESTABLISHED) apache2 14696 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14696 www-data 16u IPv6 209771 0t0 TCP fog.myserver.co.uk:http->10.1.26.163:64329 (ESTABLISHED) apache2 14697 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14697 www-data 16u IPv6 207285 0t0 TCP fog.myserver.co.uk:http->10.1.25.174:56202 (ESTABLISHED) apache2 14701 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14701 www-data 16u IPv6 207548 0t0 TCP fog.myserver.co.uk:http->10.28.7.15:63067 (ESTABLISHED) apache2 14710 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14710 www-data 16u IPv6 209927 0t0 TCP fog.myserver.co.uk:http->10.28.10.178:59324 (ESTABLISHED) apache2 14713 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14713 www-data 16u IPv6 209794 0t0 TCP fog.myserver.co.uk:http->10.28.8.20:56839 (ESTABLISHED) apache2 14714 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14714 www-data 16u IPv6 211325 0t0 TCP fog.myserver.co.uk:http->10.28.10.180:49471 (ESTABLISHED) apache2 14716 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14716 www-data 16u IPv6 207539 0t0 TCP fog.myserver.co.uk:http->10.9.106.221:49627 (ESTABLISHED) apache2 14725 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14725 www-data 16u IPv6 209585 0t0 TCP fog.myserver.co.uk:http->pc24-17.myserver.co.uk:65308 (ESTABLISHED) apache2 14726 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14726 www-data 16u IPv6 199604 0t0 TCP fog.myserver.co.uk:http->10.1.25.139:65317 (ESTABLISHED) apache2 14727 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14727 www-data 16u IPv6 215258 0t0 TCP fog.myserver.co.uk:http->10.1.21.170:29597 (ESTABLISHED) apache2 14728 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14728 www-data 16u IPv6 209451 0t0 TCP fog.myserver.co.uk:http->10.1.23.219:51341 (ESTABLISHED) apache2 14731 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14731 www-data 16u IPv6 209752 0t0 TCP fog.myserver.co.uk:http->10.9.106.32:49646 (ESTABLISHED) apache2 14734 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14734 www-data 16u IPv6 206560 0t0 TCP fog.myserver.co.uk:http->10.1.25.143:50171 (ESTABLISHED) apache2 14735 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14735 www-data 16u IPv6 208701 0t0 TCP fog.myserver.co.uk:http->10.9.106.26:50250 (ESTABLISHED) apache2 14748 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14748 www-data 16u IPv6 206878 0t0 TCP fog.myserver.co.uk:http->10.1.25.135:53350 (ESTABLISHED) apache2 14756 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14756 www-data 16u IPv6 205462 0t0 TCP fog.myserver.co.uk:http->10.240.11.232:49580 (ESTABLISHED) apache2 14757 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 14757 www-data 16u IPv6 209305 0t0 TCP fog.myserver.co.uk:http->10.240.11.36:49789 (ESTABLISHED) apache2 15211 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15211 www-data 16u IPv6 205319 0t0 TCP fog.myserver.co.uk:http->10.28.10.194:62332 (ESTABLISHED) apache2 15234 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15234 www-data 16u IPv6 199457 0t0 TCP fog.myserver.co.uk:http->10.9.106.45:50690 (ESTABLISHED) apache2 15235 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15235 www-data 16u IPv6 208384 0t0 TCP fog.myserver.co.uk:http->10.1.26.171:50388 (ESTABLISHED) apache2 15282 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15282 www-data 16u IPv6 205353 0t0 TCP fog.myserver.co.uk:http->10.9.106.31:49660 (ESTABLISHED) apache2 15307 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15307 www-data 16u IPv6 199602 0t0 TCP fog.myserver.co.uk:http->10.28.11.207:52033 (ESTABLISHED) apache2 15308 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15308 www-data 16u IPv6 204923 0t0 TCP fog.myserver.co.uk:http->10.9.112.166:56260 (ESTABLISHED) apache2 15352 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15352 www-data 16u IPv6 211225 0t0 TCP fog.myserver.co.uk:http->10.240.11.199:49886 (ESTABLISHED) apache2 15367 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15367 www-data 16u IPv6 209694 0t0 TCP fog.myserver.co.uk:http->pc26-25.myserver.co.uk:52138 (ESTABLISHED) apache2 15370 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15370 www-data 16u IPv6 209280 0t0 TCP fog.myserver.co.uk:http->10.1.25.156:53894 (ESTABLISHED) apache2 15371 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15371 www-data 16u IPv6 207943 0t0 TCP fog.myserver.co.uk:http->10.240.11.232:49564 (ESTABLISHED) apache2 15385 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15385 www-data 16u IPv6 206583 0t0 TCP fog.myserver.co.uk:http->10.9.106.36:49647 (ESTABLISHED) apache2 15389 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15389 www-data 16u IPv6 211236 0t0 TCP fog.myserver.co.uk:http->10.1.24.160:52021 (ESTABLISHED) apache2 15390 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15390 www-data 16u IPv6 209116 0t0 TCP fog.myserver.co.uk:http->10.1.23.197:50257 (ESTABLISHED) apache2 15393 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15393 www-data 16u IPv6 209989 0t0 TCP fog.myserver.co.uk:http->10.1.26.224:56221 (ESTABLISHED) apache2 15396 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15396 www-data 16u IPv6 209685 0t0 TCP fog.myserver.co.uk:http->10.28.10.24:55542 (ESTABLISHED) apache2 15399 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15399 www-data 16u IPv6 211314 0t0 TCP fog.myserver.co.uk:http->10.1.23.237:56843 (ESTABLISHED) apache2 15789 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15789 www-data 16u IPv6 208833 0t0 TCP fog.myserver.co.uk:http->10.28.7.35:49378 (ESTABLISHED) apache2 15790 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15790 www-data 16u IPv6 207953 0t0 TCP fog.myserver.co.uk:http->10.28.10.95:58542 (ESTABLISHED) apache2 15793 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15793 www-data 16u IPv6 206758 0t0 TCP fog.myserver.co.uk:http->10.1.26.105:50066 (ESTABLISHED) apache2 15794 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15794 www-data 16u IPv6 205274 0t0 TCP fog.myserver.co.uk:http->10.1.23.216:56813 (ESTABLISHED) apache2 15795 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15795 www-data 16u IPv6 206577 0t0 TCP fog.myserver.co.uk:http->10.1.26.94:62086 (ESTABLISHED) apache2 15803 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15803 www-data 16u IPv6 207925 0t0 TCP fog.myserver.co.uk:http->10.28.10.219:64208 (ESTABLISHED) apache2 15804 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15804 www-data 16u IPv6 209654 0t0 TCP fog.myserver.co.uk:http->10.240.11.36:49790 (ESTABLISHED) apache2 15806 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15806 www-data 16u IPv6 205341 0t0 TCP fog.myserver.co.uk:http->10.1.55.226:53041 (ESTABLISHED) apache2 15807 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15807 www-data 16u IPv6 208059 0t0 TCP fog.myserver.co.uk:http->10.9.106.34:49648 (ESTABLISHED) apache2 15808 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15808 www-data 16u IPv6 206831 0t0 TCP fog.myserver.co.uk:http->10.1.26.93:63211 (ESTABLISHED) apache2 15809 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15809 www-data 16u IPv6 209494 0t0 TCP fog.myserver.co.uk:http->10.28.10.203:55371 (FIN_WAIT2) apache2 15813 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15813 www-data 16u IPv6 207512 0t0 TCP fog.myserver.co.uk:http->10.28.10.93:49609 (ESTABLISHED) apache2 15814 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15814 www-data 16u IPv6 209625 0t0 TCP fog.myserver.co.uk:http->10.240.11.226:49533 (ESTABLISHED) apache2 15815 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15815 www-data 16u IPv6 199544 0t0 TCP fog.myserver.co.uk:http->10.1.26.88:49757 (ESTABLISHED) apache2 15816 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15816 www-data 16u IPv6 209879 0t0 TCP fog.myserver.co.uk:http->10.1.55.228:49738 (ESTABLISHED) apache2 15817 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15817 www-data 16u IPv6 208973 0t0 TCP fog.myserver.co.uk:http->10.28.10.168:60392 (ESTABLISHED) apache2 15818 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15818 www-data 16u IPv6 206574 0t0 TCP fog.myserver.co.uk:http->10.28.10.203:55369 (ESTABLISHED) apache2 15819 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15819 www-data 16u IPv6 210971 0t0 TCP fog.myserver.co.uk:http->10.28.12.225:50235 (ESTABLISHED) apache2 15820 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15820 www-data 16u IPv6 208821 0t0 TCP fog.myserver.co.uk:http->10.1.28.51:50042 (ESTABLISHED) apache2 15833 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15833 www-data 16u IPv6 208149 0t0 TCP fog.myserver.co.uk:http->10.28.10.11:54096 (ESTABLISHED) apache2 15837 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15837 www-data 16u IPv6 207433 0t0 TCP fog.myserver.co.uk:http->pc24-23.myserver.co.uk:50093 (ESTABLISHED) apache2 15927 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15927 www-data 16u IPv6 206566 0t0 TCP fog.myserver.co.uk:http->10.1.26.60:50475 (ESTABLISHED) apache2 15928 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15928 www-data 16u IPv6 205446 0t0 TCP fog.myserver.co.uk:http->10.28.10.157:63434 (ESTABLISHED) apache2 15929 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 15929 www-data 16u IPv6 206629 0t0 TCP fog.myserver.co.uk:http->10.28.10.199:54524 (ESTABLISHED) apache2 16107 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 16107 www-data 16u IPv6 208099 0t0 TCP fog.myserver.co.uk:http->10.28.10.14:50199 (ESTABLISHED) apache2 17032 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17032 www-data 16u IPv6 207327 0t0 TCP fog.myserver.co.uk:http->10.28.10.192:51399 (ESTABLISHED) apache2 17311 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17311 www-data 16u IPv6 207968 0t0 TCP fog.myserver.co.uk:http->10.240.11.153:49295 (ESTABLISHED) apache2 17315 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17315 www-data 16u IPv6 208474 0t0 TCP fog.myserver.co.uk:http->10.9.106.27:49633 (ESTABLISHED) apache2 17316 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17316 www-data 16u IPv6 206688 0t0 TCP fog.myserver.co.uk:http->10.9.110.165:56298 (ESTABLISHED) apache2 17320 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17320 www-data 16u IPv6 209074 0t0 TCP fog.myserver.co.uk:http->10.1.24.172:51523 (ESTABLISHED) apache2 17321 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17321 www-data 16u IPv6 208433 0t0 TCP fog.myserver.co.uk:http->10.1.25.151:50002 (ESTABLISHED) apache2 17322 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17322 www-data 16u IPv6 206784 0t0 TCP fog.myserver.co.uk:http->10.28.12.225:50233 (ESTABLISHED) apache2 17323 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17323 www-data 16u IPv6 207321 0t0 TCP fog.myserver.co.uk:http->10.1.25.178:58762 (ESTABLISHED) apache2 17328 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17328 www-data 16u IPv6 206844 0t0 TCP fog.myserver.co.uk:http->10.1.23.239:50334 (ESTABLISHED) apache2 17329 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17329 www-data 16u IPv6 199659 0t0 TCP fog.myserver.co.uk:http->10.240.11.235:49983 (ESTABLISHED) apache2 17330 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17330 www-data 16u IPv6 205477 0t0 TCP fog.myserver.co.uk:http->10.28.10.177:59575 (ESTABLISHED) apache2 17335 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17335 www-data 16u IPv6 208329 0t0 TCP fog.myserver.co.uk:http->10.1.25.129:52700 (ESTABLISHED) apache2 17336 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17336 www-data 16u IPv6 209230 0t0 TCP fog.myserver.co.uk:http->10.1.37.27:27513 (ESTABLISHED) apache2 17337 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 17337 www-data 16u IPv6 199409 0t0 TCP fog.myserver.co.uk:http->10.240.11.182:55925 (ESTABLISHED) apache2 18513 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18513 www-data 16u IPv6 208128 0t0 TCP fog.myserver.co.uk:http->10.1.26.166:63340 (ESTABLISHED) apache2 18519 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18519 www-data 16u IPv6 211274 0t0 TCP fog.myserver.co.uk:http->10.1.25.119:50908 (ESTABLISHED) apache2 18525 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18525 www-data 16u IPv6 207974 0t0 TCP fog.myserver.co.uk:http->10.240.11.199:49885 (ESTABLISHED) apache2 18526 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18526 www-data 16u IPv6 208623 0t0 TCP fog.myserver.co.uk:http->pc24-31.myserver.co.uk:49703 (ESTABLISHED) apache2 18530 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18530 www-data 16u IPv6 209554 0t0 TCP fog.myserver.co.uk:http->10.28.10.155:56252 (ESTABLISHED) apache2 18534 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18534 www-data 16u IPv6 205313 0t0 TCP fog.myserver.co.uk:http->10.28.10.60:52758 (ESTABLISHED) apache2 18535 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18535 www-data 16u IPv6 207876 0t0 TCP fog.myserver.co.uk:http->10.9.106.43:49624 (ESTABLISHED) apache2 18542 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18542 www-data 16u IPv6 211385 0t0 TCP fog.myserver.co.uk:http->10.28.11.223:50357 (ESTABLISHED) apache2 18555 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18555 www-data 16u IPv6 206477 0t0 TCP fog.myserver.co.uk:http->10.1.55.107:51872 (FIN_WAIT2) apache2 18560 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18560 www-data 16u IPv6 206707 0t0 TCP fog.myserver.co.uk:http->10.28.12.239:52158 (ESTABLISHED) apache2 18561 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18561 www-data 16u IPv6 209474 0t0 TCP fog.myserver.co.uk:http->10.1.26.166:63341 (FIN_WAIT2) apache2 18564 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18564 www-data 16u IPv6 206586 0t0 TCP fog.myserver.co.uk:http->10.28.7.60:49773 (ESTABLISHED) apache2 18565 www-data 4u IPv6 12460 0t0 TCP *:http (LISTEN) apache2 18565 www-data 16u IPv6 208156 0t0 TCP fog.myserver.co.uk:http->10.28.10.92:58723 (ESTABLISHED) root@fog:/var/log/mysql#Will keep an eye on what people do
