database update issues following image capture
-
Good morning,
e have been fighting an issue with the FOG database updating following an image capture.
Previously, it was a memory issue and we have corrected that.However, now we have a new error:
Error returned: <!DOCTYPE HTML PUBLIC “-//IETF//DD HTML //EN”>
<html><head>
<title Gateway Timeout</title>
</head><body>
<h1>Gateway Timeout </h1>
<p>The gateway did not receive a timely response from the upstream server or application.
</p>
<hr>
<address>Apache/ (Ubuntu) Server at address 192.168.3.2 Port 80</address>
</body></html>Not being real experienced with Linux, I’m not sure the location of logs I should check for an issue. We know the apache server is running and the website is up, as we can access the site to deploy images w/o issue. Additionally, I am able to register and inventory new machines w/o issue. It is only when trying to capture an image to have a newer version of it that we run into problems.
FOG version, from the site, listed as 1.5.9.119
Ubuntu version 20.04.4 LTS- All updates/upgrades current
Thanks.
-
@epatterson Did the error spit out html or is that an artifact of copy and paste?
The gateway its referring to is php-fpm. That is the back end processor that processes php code.
You should see php-fpm is running by
ls -la | grep php-fpmThere should be at least one process running, there will / should be at least 5 running with that name. -
@george1421
Thanks for the response.
The error I entered is directly from the terminal when it is finishing up the imaged capture. It tries to update the database, fails, then loops through continually until stopped. -
@george1421
Do I need to run the command while an image capture is running? I get no results if I just run it in the terminal currently. -
@epatterson said in database update issues following image capture:
I get no results if I just run it in the terminal currently
php-fpm should be running in the background like apache is.
systemctl status php-fpmWIth ubuntu the name may not be php-fpm but php-fpm7.4 or what ever version of php that is installed. I don’t use ubuntu so I can’t give you an exact answer. -
@george1421
Yup. I had seen php7.4-fpm.service in the results of >systemctl list-units --type=service earlier.
I wasn’t sure if that was the match needed.
After altering the command from earlier, I still get no results from itls -al | grep php7.4-fpm
or
ls -al | grep php7.4-fpm.service
The command just completes w/o any results displayed, dropping down to a fresh line in the terminal -
@epatterson said in database update issues following image capture:
php7.4-fpm
Ok then what is the output of this
systemctl status php7.4-fpmThe php-fpm engine needs to be running.also
systemctl restart php7.4-fpmThen run a status check.FWIW the
ls -alcommand as you have it below is list directory notpsprocess status. Your command will look for a file in the current directory called php7.4-fpm -
@george1421
It is listed as:
Loaded: loaded
Active: active (running)
Status: Processes Active
Server was rebooted this morning, and has been rebooted previously when receiving the error to see if that would kick it into gear -
@george1421
Thought the actual output might be more helpful:
sysadmin@fog:/$ systemctl status php7.4-fpm
● php7.4-fpm.service - The PHP 7.4 FastCGI Process Manager
Loaded: loaded (/lib/systemd/system/php7.4-fpm.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2022-03-25 10:11:58 CDT; 1h 7min ago
Docs: man:php-fpm7.4(8)
Process: 1141 ExecStartPost=/usr/lib/php/php-fpm-socket-helper install /run/php/php-fpm.sock /etc/php/7.4/fpm/pool.d/www.conf 74 (code=exited, st>
Main PID: 863 (php-fpm7.4)
Status: “Processes active: 0, idle: 6, Requests: 68, slow: 0, Traffic: 0req/sec”
Tasks: 7 (limit: 14143)
Memory: 28.9M
CGroup: /system.slice/php7.4-fpm.service
├─ 863 php-fpm: master process (/etc/php/7.4/fpm/php-fpm.conf)
├─1136 php-fpm: pool www
├─1137 php-fpm: pool www
├─1138 php-fpm: pool www
├─1139 php-fpm: pool www
├─1140 php-fpm: pool www
└─1556 php-fpm: pool wwwMar 25 10:11:47 fog.it.nex-tech.com systemd[1]: Starting The PHP 7.4 FastCGI Process Manager…
Mar 25 10:11:58 fog.it.nex-tech.com systemd[1]: Started The PHP 7.4 FastCGI Process Manager. -
@epatterson OK now that we’ve determined that php-fpm is running, can you access the web site? And to recap everything works OK except its failing to update the database at the end of image deployment? The inistial error you posted is apache saying hey I can’t talk to php-fpm.
We may need to consult either the apache or php-fpm error log right after this error message happens to see what is getting posted in the logs.
-
@george1421
During the process, we have not had any issue accessing the site. I am able to PXE boot and register/inventory machines w/o issue. I am able to deploy an image w/o issue and monitor the process on the site from Active Tasks.Additionally, when running a capture, I am able to monitor it on the site as well. I am testing a capture now, to get a recent log entry to look into. I may need some guidance locating the logs, as I am still learning linux. I know I have been in them before, when we had the memory issue with PHP.
-
@epatterson logs are in /var/logs directory. There is the apache directory and php-fpm may have its own directory or just in /var/logs.
-
@george1421
Well, holy mother of all that is good and green. The machine I just tested with completed w/o issue.Could it have anything to do with the model, or image itself? We normally work with Lenovo T15 and/or T14 machines. However, I have an X1 Carbon that we needed to wipe prior to being sent in for repair, so I registered and inventoried it. Created a new image category, specifically for it, so we can put it back when it returns, and ran the capture.
I have another machine (T15) that I just built. I’ll try capturing it as well and see what happens. I’ll update the thread once done. Thanks for helping me out today.
-
@epatterson said in database update issues following image capture:
Could it have anything to do with the model, or image itself?
Don’t think so. The gateway timeout is usually not specific to a particular model. Maybe there was a flood of hosts sending requests (fog-client installed) at some point that is not happening anymore now.
-
@sebastian-roth
OK. I will be doing some more testing next week. I will update the thread with results. Thanks. -
@epatterson said in database update issues following image capture:
I have another machine (T15) that I just built. I’ll try capturing it as well and see what happens. I’ll update the thread once done. Thanks for helping me out today.
My concern is that we didn’t fix anything, so we really did solve the problem. It went away by itself and most likely will return without warning.
-
@sebastian-roth
Good morning,
I imaged some new machines yesterday, updated their applications, etc, and ran captures, on 2 of them, back to FOG just now.
The image capture ran correctly, but the process, again, failed on the database update. Receiving a “504 Gateway timeout” again.
The successful capture Friday was on a completely new image, first time capture.
I will try looking into logs later today. -
@epatterson So does the apache or php-fpm error logs give us any clue to why this is happening?
Out of curiosity, how many computers with the fog client installed is this fog server talking to? Or to ask it a different way, how many computer have the fog client installed on it?
-
@george1421
Our setup is a single FOG server, with 2 network connections.
One to the main domain, and a second to a closed network that we use when imaging machines.
We aren’t actually installing a client on the machines. They are only connected to the FOG closed network to be imaged, or to capture an image back to FOG once updated.I am running a test currently. I have imaged a machine, I am updating it, and will create a new image category to capture the image back to the system. I’ll update the thread once I have tried imaging back to the system. Thanks.
-
@epatterson Ok that is fine. I was just trying to think of a reason why on occasion we get this gateway timeout. Having 500 computers checking into the fog server while trying to image might do just that. But if you don’t have the fog client running anywhere then fog server performance isn’t the problem. The rest of your setup sounds typical for a business environment.
