Attempting To Send Inventory - Invalid Host? SVN 4193

Tom Elliott

Can you get the “invalid host” and immediately get a tail of your apache log? It will be extremely helpful in figuring out the issue.

Raymond Bell

@Tom-Elliott

[Wed Oct 21 12:39:48.260029 2015] [:error] [pid 17732] [client 10.24.28.18:60773] PHP Warning:  mysqli::mysqli(): (HY000/2002): Connection refused in /var/www/html/fog/lib/db/MySQL.class.php on line 37
[Wed Oct 21 12:39:48.260368 2015] [:error] [pid 17732] [client 10.24.28.18:60773] PHP Warning:  mysqli::mysqli(): (HY000/2002): Connection refused in /var/www/html/fog/lib/db/MySQL.class.php on line 39
[Wed Oct 21 12:39:48.260503 2015] [:error] [pid 17732] [client 10.24.28.18:60773] PHP Warning:  mysqli::query(): Couldn't fetch mysqli in /var/www/html/fog/lib/db/MySQL.class.php on line 63
[Wed Oct 21 12:39:48.270557 2015] [:error] [pid 17732] [client 10.24.28.18:60773] PHP Warning:  mysqli::mysqli(): (HY000/2002): Connection refused in /var/www/html/fog/lib/db/MySQL.class.php on line 37
[Wed Oct 21 12:39:48.270902 2015] [:error] [pid 17732] [client 10.24.28.18:60773] PHP Warning:  mysqli::mysqli(): (HY000/2002): Connection refused in /var/www/html/fog/lib/db/MySQL.class.php on line 39
[Wed Oct 21 12:39:48.271034 2015] [:error] [pid 17732] [client ```

Tom Elliott

Well that’s the issue. It isn’t the invalid host problem. It’s hitting the load to the mysql server. My guess is your system load is huge right now…but why, what is causing this? I don’t know that is why I ask. It’s nothing you’re doing, or have done. It’s something I did but I don’t know what.

Raymond Bell

@Tom-Elliott you will get it worked out you always do…

Sebastian Roth

@Raymond-Bell Maybe you can help!? Could you please connect to your database via command line and run a query to see what is going on in the database:

shell> mysql -u root -p fog
Enter password:
...
mysql> show full processlist;
...

Maybe use a different user than ‘root’ depending on how you configured your mysql db. Please post what you see in the processlist here in the forum. For some details see Wayne Workman’s post here: https://forums.fogproject.org/topic/5925/after-update-to-svn-4928-web-pages-time-out-constantly/44

Raymond Bell

@Uncle-Frank

mysql> show full processlist;
+-----+------+-----------------+------+---------+------+-------+-----------------------+
| Id  | User | Host            | db   | Command | Time | State | Info                  |
+-----+------+-----------------+------+---------+------+-------+-----------------------+
|   1 | root | localhost:43411 | fog  | Sleep   |   59 |       | NULL                  |
|   3 | root | localhost:43412 | fog  | Sleep   |  419 |       | NULL                  |
| 581 | root | localhost:50244 | fog  | Sleep   |    4 |       | NULL                  |
| 595 | root | localhost:50345 | fog  | Sleep   |    8 |       | NULL                  |
| 610 | root | localhost:50528 | fog  | Sleep   |    6 |       | NULL                  |
| 617 | root | localhost:50658 | fog  | Sleep   |    9 |       | NULL                  |
| 633 | root | localhost:50682 | fog  | Sleep   |    5 |       | NULL                  |
| 659 | root | localhost:50708 | fog  | Sleep   |    3 |       | NULL                  |
| 660 | root | localhost:50709 | fog  | Sleep   |    6 |       | NULL                  |
| 662 | root | localhost:50711 | fog  | Sleep   |    3 |       | NULL                  |
| 674 | root | localhost:50723 | fog  | Sleep   |    2 |       | NULL                  |
| 676 | root | localhost:50725 | fog  | Sleep   |    7 |       | NULL                  |
| 700 | root | localhost:51600 | fog  | Sleep   |    0 |       | NULL                  |
| 702 | root | localhost:52100 | fog  | Sleep   |    4 |       | NULL                  |
| 704 | root | localhost:52243 | fog  | Sleep   |    5 |       | NULL                  |
| 705 | root | localhost:52313 | fog  | Sleep   |    8 |       | NULL                  |
| 706 | root | localhost:52316 | fog  | Sleep   |    2 |       | NULL                  |
| 708 | root | localhost:52336 | fog  | Sleep   |    0 |       | NULL                  |
| 709 | root | localhost:52343 | fog  | Sleep   |    6 |       | NULL                  |
| 710 | root | localhost:52386 | fog  | Sleep   |    6 |       | NULL                  |
| 711 | root | localhost:52393 | fog  | Sleep   |    9 |       | NULL                  |
| 712 | root | localhost:52402 | fog  | Sleep   |    0 |       | NULL                  |
| 713 | root | localhost:52405 | fog  | Sleep   |    2 |       | NULL                  |
| 714 | root | localhost:52406 | fog  | Sleep   |    0 |       | NULL                  |
| 715 | root | localhost:52414 | fog  | Sleep   |    0 |       | NULL                  |
| 716 | root | localhost:52423 | fog  | Sleep   |    9 |       | NULL                  |
| 717 | root | localhost:52425 | fog  | Sleep   |    3 |       | NULL                  |
| 718 | root | localhost:52426 | fog  | Sleep   |    8 |       | NULL                  |
| 720 | root | localhost:52430 | fog  | Sleep   |    0 |       | NULL                  |
| 721 | root | localhost:52431 | fog  | Sleep   |    5 |       | NULL                  |
| 722 | root | localhost:52432 | fog  | Sleep   |    9 |       | NULL                  |
| 723 | root | localhost:52439 | fog  | Sleep   |    8 |       | NULL                  |
| 724 | root | localhost:52441 | fog  | Sleep   |    4 |       | NULL                  |
| 726 | root | localhost:52448 | fog  | Sleep   |    4 |       | NULL                  |
| 727 | root | localhost:52455 | fog  | Sleep   |    4 |       | NULL                  |
| 728 | root | localhost:52459 | fog  | Sleep   |    1 |       | NULL                  |
| 729 | root | localhost:52461 | fog  | Sleep   |    3 |       | NULL                  |
| 730 | root | localhost:52485 | fog  | Sleep   |    5 |       | NULL                  |
| 731 | root | localhost:52495 | fog  | Sleep   |    5 |       | NULL                  |
| 732 | root | localhost:52527 | fog  | Sleep   |    0 |       | NULL                  |
| 733 | root | localhost       | fog  | Query   |    0 | NULL  | show full processlist |
| 734 | root | localhost:52607 | fog  | Sleep   |    4 |       | NULL                  |
| 735 | root | localhost:52610 | fog  | Sleep   |    2 |       | NULL                  |
| 736 | root | localhost:52612 | fog  | Sleep   |    2 |       | NULL                  |
| 737 | root | localhost:52618 | fog  | Sleep   |    1 |       | NULL                  |
| 738 | root | localhost:52619 | fog  | Sleep   |    2 |       | NULL                  |
| 739 | root | localhost:52620 | fog  | Sleep   |    1 |       | NULL                  |
| 740 | root | localhost:52622 | fog  | Sleep   |    0 |       | NULL                  |
| 741 | root | localhost:52625 | fog  | Sleep   |    0 |       | NULL                  |
+-----+------+-----------------+------+---------+------+-------+-----------------------+
49 rows in set (0.00 sec)

Tom Elliott

@Raymond-Bell can you do me a favor? Can you check the /var/www/fog/lib/fog/Config.class.php file and adjust the DATABASE_HOST to say p:127.0.0.1 or if the p: is already there remove it and restart the system? Do you still see issues?

Raymond Bell

@Tom-Elliott Same issue after changing DATABASE_HOST and rebooting

Tom Elliott

@Raymond-Bell did you remove or add the p:?

Raymond Bell

@Tom-Elliott removed

Tom Elliott

Well damn.

Sebastian Roth

Why SLEEP?? I greped through the source code but could not find the mysql sleep used anywhere… Or does it mean that the connection is sleeping (not being used)?

Tom Elliott

@Uncle-Frank all the sleep means is the connection is waiting. This is because in most cases persistent connections are much better than create/use/close methods.

Sebastian Roth

@Tom-Elliott For sure persistent connections are better than opening a new one for every call. But how do you keep track of all the open connections? Why are there several dozens of open connections? Re-using a persistent connection is one thing. But keeping “hundreds” of them?

Tom Elliott

@Uncle-Frank that’s the fun bit. I don’t and it only recently started occurring.

Sebastian Roth

@Tom-Elliott And I guess you can’t reproduce this on your system?!

@Raymond-Bell How many clients do you have? Could you please run this command on your FOG server:

lsof -i :3306
COMMAND   PID     USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
apache2 22534 www-data   12u  IPv4 21769312      0t0  TCP localhost:43411->localhost:mysql (ESTABLISHED)
mysqld  24347    mysql   10u  IPv4 20564983      0t0  TCP localhost:mysql (LISTEN)
mysqld  24347    mysql   31u  IPv4 21769313      0t0  TCP localhost:mysql->localhost:43411 (ESTABLISHED)

Maybe you need to install it first. Package in debian is called ‘lsof’ just like the tool itself.

Tom Elliott

@Uncle-Frank of course I can’t. I believe you’re on the right track though as I only have one or two clients connecting to my system. And most of the time they are off to begin with. So it isn’t a surprise to me if it is related to sheer numbers of clients connecting and disconnecting. I’m of the mind though that it’s my attempt to use the real connect after initializing the mysqli object in lib/db/MySQL.class.php and missing some check somewhere as this is a fairly new issue, not one people have been experiencing the whole time.

Sebastian Roth

Searching through the forums I saw that Raymond Bell and Joseph Hales both mentioned those errors in apache log some time ago. For example see here: https://forums.fogproject.org/topic/5661/web-interface-slow-svn-4333

Browsing the code history around that time I found this: http://sourceforge.net/p/freeghost/code/3971/

Why a connect call (in the query function) which wasn’t there before?

Tom Elliott

@Uncle-Frank The reasoning is that in some cases the connection just appears to go dead. So on the construct (which is already called) it automatically makes the link, rather than implicitly making it do so.

Sebastian Roth

@Raymond-Bell And while you are at it. Please run the same command using port 80 again. My guess is that you see a lot of apache processes in the first listing (port 3306) and it would be interesting to see “who” is calling all these (connections from your clients is my guess): lsof -i :80

Attempting To Send Inventory - Invalid Host? SVN 4193

230

12.0k

17.3k

155.2k