Stuck on : "Ensuring node username and passwords match...."

[plegrand]

Hello,
i just updated my fog server with 1.5.9 version on debian .
Everything seems to works fine . Just an error message at the end during “Ensuring node username and passwords match…”

The following error was encountered while trying to retrieve the URL: http://192.168.39.243/fog/maintenance/create_update_node.php

    Connection to 192.168.39.243 failed.

The system returned: (110) Connection timed out

The remote host or network may be down. Please try the request again.

Is there a way to know what the installer try to do during this step ?

When i do a “wget http://192.168.39.243/fog/maintenance/create_update_node.php” on the server it works fine

Thanks for your help
Pascal

[Sebastian Roth]

@plegrand It’s strange it wasn’t able to send that request within the installer. Please take a look at the apache error log (see my signature) to see if there is something related at the same time your ran this.

[plegrand]

I think these errors are consistent with when I was updating :

[Tue Oct 13 15:19:45.400239 2020] [proxy_fcgi:error] [pid 18237] [client 192.168.39.202:59804] AH01067: Failed to read FastCGI header
[Tue Oct 13 15:19:45.400269 2020] [proxy_fcgi:error] [pid 18237] (104)Connection reset by peer: [client 192.168.39.202:59804] AH01075: Error dispatching request to :
[Tue Oct 13 15:19:45.424226 2020] [proxy:error] [pid 23571] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:19:45.424272 2020] [proxy_fcgi:error] [pid 23571] [client 192.168.39.202:59805] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:19:45.426384 2020] [proxy:error] [pid 18238] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:19:45.426412 2020] [proxy_fcgi:error] [pid 18238] [client 192.168.39.202:59806] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:19:45.447120 2020] [mpm_prefork:notice] [pid 707] AH00169: caught SIGTERM, shutting down
[Tue Oct 13 15:20:27.916679 2020] [mpm_prefork:notice] [pid 13584] AH00163: Apache/2.4.38 (Debian) OpenSSL/1.1.1d configured -- resuming normal operations
[Tue Oct 13 15:20:27.916729 2020] [core:notice] [pid 13584] AH00094: Command line: '/usr/sbin/apache2'
[Tue Oct 13 15:20:27.993053 2020] [proxy:error] [pid 13585] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:20:27.993095 2020] [proxy_fcgi:error] [pid 13585] [client 192.168.39.201:59662] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:20:28.043766 2020] [proxy:error] [pid 13586] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:20:28.043797 2020] [proxy_fcgi:error] [pid 13586] [client 192.168.39.144:49574] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:20:28.092625 2020] [proxy:error] [pid 13587] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:20:28.092655 2020] [proxy_fcgi:error] [pid 13587] [client 192.168.39.144:49575] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:20:28.095287 2020] [proxy:error] [pid 13588] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:20:28.095315 2020] [proxy_fcgi:error] [pid 13588] [client 192.168.39.144:49576] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:20:28.097553 2020] [proxy:error] [pid 13589] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:20:28.097580 2020] [proxy_fcgi:error] [pid 13589] [client 192.168.39.201:59663] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:20:28.100708 2020] [proxy:error] [pid 13585] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:20:28.100727 2020] [proxy_fcgi:error] [pid 13585] [client 192.168.39.201:59664] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:20:28.119168 2020] [proxy:error] [pid 13586] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:20:28.119194 2020] [proxy_fcgi:error] [pid 13586] [client 192.168.39.83:50776] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:20:28.152719 2020] [proxy:error] [pid 13587] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:20:28.152734 2020] [proxy_fcgi:error] [pid 13587] [client 192.168.39.51:51557] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:20:28.167072 2020] [proxy:error] [pid 13588] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:20:28.167087 2020] [proxy_fcgi:error] [pid 13588] [client 192.168.39.83:50777] AH01079: failed to make connection to backend: 127.0.0.1
[Tue Oct 13 15:20:28.170549 2020] [proxy:error] [pid 13589] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:9000 (*) failed
[Tue Oct 13 15:20:28.170563 2020] [proxy_fcgi:error] [pid 13589] [client 192.168.39.83:50778] AH01079: failed to make connection to backend: 127.0.0.1

[Sebastian Roth]

@plegrand Does the FOG web UI work at all?

Please run dpkg -l | grep php and post output here.

[plegrand]

@Sebastian-Roth said in Stuck on : "Ensuring node username and passwords match....":

dpkg -l | grep php

[root@fog] ~ # dpkg -l | grep php
ii  libapache2-mod-php7.3          7.3.19-1~deb10u1              amd64        server-side, HTML-embedded scripting language (Apache 2 module)
ii  php                            2:7.3+69                      all          server-side, HTML-embedded scripting language (default)
ii  php-cli                        2:7.3+69                      all          command-line interpreter for the PHP scripting language (default)
ii  php-common                     2:69                          all          Common files for PHP packages
ii  php-curl                       2:7.3+69                      all          CURL module for PHP [default]
ii  php-fpm                        2:7.3+69                      all          server-side, HTML-embedded scripting language (FPM-CGI binary) (default)
ii  php-gettext                    1.0.12-0.1                    all          transitional dummy package for php-php-gettext
ii  php-mbstring                   2:7.3+69                      all          MBSTRING module for PHP [default]
ii  php-mysql                      2:7.3+69                      all          MySQL module for PHP [default]
ii  php-pear                       1:1.10.6+submodules+notgz-1.1 all          PEAR Base System
ii  php-php-gettext                1.0.12-0.1                    all          read gettext MO files directly, without requiring anything other than PHP
ii  php-xml                        2:7.3+69                      all          DOM, SimpleXML, WDDX, XML, and XSL module for PHP [default]
ii  php7.3                         7.3.19-1~deb10u1              all          server-side, HTML-embedded scripting language (metapackage)
ii  php7.3-bcmath                  7.3.19-1~deb10u1              amd64        Bcmath module for PHP
ii  php7.3-cli                     7.3.19-1~deb10u1              amd64        command-line interpreter for the PHP scripting language
ii  php7.3-common                  7.3.19-1~deb10u1              amd64        documentation, examples and common module for PHP
ii  php7.3-curl                    7.3.19-1~deb10u1              amd64        CURL module for PHP
ii  php7.3-fpm                     7.3.19-1~deb10u1              amd64        server-side, HTML-embedded scripting language (FPM-CGI binary)
ii  php7.3-gd                      7.3.19-1~deb10u1              amd64        GD module for PHP
ii  php7.3-json                    7.3.19-1~deb10u1              amd64        JSON module for PHP
ii  php7.3-mbstring                7.3.19-1~deb10u1              amd64        MBSTRING module for PHP
ii  php7.3-mysql                   7.3.19-1~deb10u1              amd64        MySQL module for PHP
ii  php7.3-opcache                 7.3.19-1~deb10u1              amd64        Zend OpCache module for PHP
ii  php7.3-readline                7.3.19-1~deb10u1              amd64        readline module for PHP
ii  php7.3-xml                     7.3.19-1~deb10u1              amd64        DOM, SimpleXML, WDDX, XML, and XSL module for PHP

[plegrand]

@Sebastian-Roth The FOG UI seems to work fine

[plegrand]

@Sebastian-Roth
Is it safe t “re run” the installer to see exactly the good logs ?
Is there other logs to watch except /var/log/apache2/error.log ?

Can you tell me what the installer try to do?
A wget request?
with wich url ?

[Sebastian Roth]

@plegrand Looks ok to me so far.

Is it safe t “re run” the installer to see exactly the good logs ?

Yes, you can re-run the installer. Probably a good idea to see if it’s still happening.

Is there other logs to watch except /var/log/apache2/error.log ?

That log is fine. You might want to check the PHP-FPM log as well (see my signature again).

Can you tell me what the installer try to do?

You can see it’s just a curl request to the FOG webserver to make sure the storage node settings (username/password/IP) in the database are correct.

[plegrand]

@Sebastian-Roth
Has it always been curl?
I’m behind a proxy and i made this configuration for wget :
http_proxy = http://cache.blabla.fr:3128/
https_proxy = http://cache.blabla.fr:3128/
use_proxy = on
no_proxy=localhost,127.0.0.0/8,192.168.39.243
May be i should make the same configuration for curl ?

I just remember that i made an export comand for proxy, then installer use proxy for all actions.
I’ll test tomorrow whithout export command to see.
I’ll tell you

[Sebastian Roth]

@plegrand said in Stuck on : "Ensuring node username and passwords match....":

Has it always been curl?

That is a good question. Let me check the code.

Yeah right, we changed from wget to curl between 1.5.8 and 1.5.9: https://github.com/FOGProject/fogproject/commit/587a25f37b4e5daf61a337966064ffe9dfd30c15

[plegrand]

@Sebastian-Roth said in Stuck on : "Ensuring node username and passwords match....":

You can see it’s just a curl request to the FOG webserver to make sure the storage node settings (username/password/IP) in the database are correct.

$ipaddress in this line is the "real ip address of the server, not “localhost” or “127.0.0.1” ?

Thanks for your help

[plegrand]

@Sebastian-Roth

With the good config file for curl
.curlrc

proxy = http://blabla.fr:3128/
noproxy=localhost,127.0.0.0/8,192.168.39.243

It works fine.

Just for information, here is what i use :

.wgetrc

http_proxy = http://blabla.fr:3128/
https_proxy = http://blabla.fr:3128/
use_proxy = on
no_proxy=localhost,127.0.0.0/8,192.168.39.243

.gitconfig

[http]
	proxy = http://blabla.fr:3128

.curlrc

proxy = http://blabla.fr:3128/
noproxy = localhost,127.0.0.0/8,192.168.39.243

There is always a problem during this operation:

* Testing internet connection.................................There was no interface with an active internet connection found.

As we are behind proxy, the ping command does not works.
May be it could be possible to test internet connection with curl or wget ?
https://github.com/FOGProject/fogproject/blob/master/lib/common/functions.sh#L386

Thanks again

[plegrand]

@Sebastian-Roth
There is always a problem during this operation:

Testing internet connection.................................There was no interface with an active internet connection found.

As we are behind proxy, the ping command does not works.
May be it could be possible to test internet connection with curl or wget ?
https://github.com/FOGProject/fogproject/blob/master/lib/common/functions.sh#L386

Thanks again

[Sebastian Roth]

@plegrand Sorry I did not reply yet. Will surely look into using a different method to check for the internet connection. I have this in my list, don’t worry.

[Sebastian Roth]

@plegrand Testing internet connection is now done mainly with curl.