Failed to Update Database and Host

[Tom Elliott]

I’ve found and fixed several ways these empty tasks get created — a group deploy of two or more hosts was failing outright, and deleting an image was leaving its queued tasks pointing at nothing — but before I ship a cleanup for the rows you already have, I need to know whether the bad tasks hold a 0 or the id of something that got deleted, because that changes what the cleanup has to match on. Could you run this against your FOG database and paste the output?

SELECT  t.taskID,
        t.taskStateID,
        t.taskTypeID,   IF(tt.ttID   IS NULL, 'MISSING', tt.ttName)   AS type_row,
        t.taskHostID,   IF(h.hostID  IS NULL, 'MISSING', h.hostName)  AS host_row,
        t.taskImageID,  IF(i.imageID IS NULL, 'MISSING', i.imageName) AS image_row,
        t.taskName,
        t.taskCreateBy,
        t.taskCreateTime
FROM       tasks      t
LEFT JOIN  hosts      h  ON h.hostID   = t.taskHostID
LEFT JOIN  images     i  ON i.imageID  = t.taskImageID
LEFT JOIN  taskTypes  tt ON tt.ttID    = t.taskTypeID
WHERE      t.taskStateID IN (0,1,2,3)
ORDER BY   t.taskID;

SELECT  t.taskStateID,
        COUNT(*)                AS rows_total,
        SUM(h.hostID  IS NULL)  AS host_missing,
        SUM(i.imageID IS NULL)  AS image_missing,
        SUM(tt.ttID   IS NULL)  AS type_missing,
        SUM(t.taskHostID  = 0)  AS host_zero,
        SUM(t.taskImageID = 0)  AS image_zero,
        SUM(t.taskTypeID  = 0)  AS type_zero,
        MIN(t.taskCreateTime)   AS oldest,
        MAX(t.taskCreateTime)   AS newest
FROM       tasks      t
LEFT JOIN  hosts      h  ON h.hostID   = t.taskHostID
LEFT JOIN  images     i  ON i.imageID  = t.taskImageID
LEFT JOIN  taskTypes  tt ON tt.ttID    = t.taskTypeID
GROUP BY   t.taskStateID;

SELECT * FROM schemaVersion;

[maxcarpone]

@Tom-Elliott : I can do that if you tell me how to do it!

Update : I figured it out.

[Tom Elliott]

@maxcarpone I don’t know if this is directed specifically at you @maxcarpone but if you want the gist:

sudo mariadb -u root fog

Then enter the select statements provided earlier.

[maxcarpone]

@Tom-Elliott : I found a way to do it :

mysql --user=fogstorage --password=MYPASSWORDHERE -D fog

Here are the results :

[Tom Elliott]

@maxcarpone That output was exactly what I needed — thank you for running it, and for the screenshot of the failing deployment, which turned out to be the most useful thing in the thread.

Your database is fine and your schema is current. schemaVersion 286 is the right number for 1.5.10.2402; 287 only landed this morning. And your first query returning an empty set means you have no stuck tasks at all — nothing in Queued or In Progress. The blank entries you’ve been seeing are all in Complete and Cancelled, and they’re just history: 300 of those rows point at hosts and 459 at images that were deleted at some point over the nine years that table has been filling up. Nothing is wrong with them, they simply have nothing left to show.

The real bug is the one in your deployment screenshot, and it is not your FTP password. Here is what was happening:

* Task Complete
* Updating Database.....................Failed
* Error returned: Failed to update imaging log
* Reattempting to update database.......Failed
* Error returned: No Active Task found for Host: VM-W11-TEST

Only that first error is real. FOG looks up the open imaging-log row to close it by asking for the row whose finish time is empty, and a change earlier in the 1.5.10 line moved “empty” from a zero date to a real NULL. The query FOG was building for that could never match a NULL — so on every single deployment, on every install, it failed to find the row it had just created minutes earlier. It then reported “Failed to update imaging log” after it had already marked the task Complete, which is why every retry after that came back “No Active Task found for Host” and the machine sat there rebooting into an error.

That also explains your very first post: “the host is well imaged but it is not updated in FOG, information about the last image isn’t written in the database.” It wasn’t. The imaging log never closed, the task was already marked Complete, and nothing recorded that the image had landed.

I’ve reproduced it end to end and fixed it, along with three other ways a task could end up pointing at nothing: https://github.com/FOGProject/fogproject/pull/1391

Two things still outstanding on your side:

1. The two MariaDB instances are worth sorting out on their own. I don’t think they caused the above, but two servers on one box is a real problem waiting to happen and it makes everything harder to diagnose. systemctl list-units 'maria*' 'mysql*' and ss -lntp | grep -E '3306|mysql' will show what’s actually listening, and sudo mariadb -u root fog -e "SELECT COUNT(*) FROM tasks" compared against the count in the web UI will tell you whether FOG and your shell are looking at the same database.

2. “Invalid Login” on register-with-immediate-deploy I have not explained yet, and I’d like to. It’s suspicious that a brand new user worked and fog didn’t, because that rules out the typing. Two questions: does the fog user’s password contain any accented or punctuation characters, and can you log into the web UI as fog right now with that same password? If the answer to the second is yes, the problem is in how the password survives the trip from the boot menu, and that’s a different fix from the one above.

[maxcarpone]

@Tom-Elliott : Good explanation and I understood the most of it. Many thanks.

I will try to help resolving these remaining bugs.

About the MariaDB Instances. I don’t understand why there are 2 instances but I’m not the only person using this server. Are they both related to Fog? I used your codelines :

I don’t know where to find the count in the web UI?

I remember I had problems with phpmysql for several years and I had to modify a file to be able to update our FOG servers. I noticed I hadn’t have to modify that file (functions.sh) anymore. Could it be related to those MariaDB instances?

Now, speaking about invalid login led me to think the password could be the culprit because it’s started with ‘&’. I’ve just modified it and will try again and finally let you know what happened.

[maxcarpone]

Bingo!

(I tried the older password just to be sure before typing the good one)

This one was related to our password which started with ‘&’
I modified it and now, I can image I soon as I register a new host. I must say that password has always worked right from the beginning till the previous version 1734 (prior to update to 2390).

So now, everything should be sorted out after updating our server to the last commit?

(Still those instances of MariaDB but I’m no good to debug it without help)

-> I’ve made some progress

[maxcarpone]

It’s all good !

As you can see, informations are now written in the database (last image).

Many thanks for your help. And long live Fog and all of you who are involved in the project !

[Tom Elliott]

@maxcarpone You found it, and you were right. The & was the whole thing. It’s now fixed in
code, so you can put your original password back.

First, a correction to my earlier reply. I said the boot-menu password was
involved. It isn’t — the credentials you type during full registration are
entered in the FOS init and posted to service/auto.register.php, which is a
completely different path. Ignore that part of what I wrote.

What was actually wrong. Registration-with-deploy was HTML-escaping your
password before comparing it against the stored hash.

The registration handler read the username and password out of the request
after running it through a shared helper (stripAndDecode()) whose last step
is htmlspecialchars(). That is the right thing to do to a value you’re about
to print into a web page, and exactly the wrong thing to do to a value you’re
about to check against a password hash. So a password containing any of

&   <   >   "   '

reached the password check as its HTML entity form — yours was compared as
&... — and could never match. A leading or trailing space was lost too.

And your timing was exactly right. You said it worked up to 1734 and broke
by 2390. The escaping was added in 1.5.10.1830 (commit 05403ee0,
19 May 2026), which sits right inside that window. It was part of a
security-hardening change, and applying that sanitiser to a credential was an
unintended side effect of it.

That also explains everything else you saw:

• a brand new user worked, because you’d have given it a simple password
  with none of those characters in it
• fog worked in the web UI, and at service/checkcredentials.php, because
  neither of those goes anywhere near that helper
• only registration-with-deploy said “Invalid Login”

For anyone else finding this thread: it is not a keyboard-layout problem,
which is where I’d have looked next. Accented characters pass through untouched
— they aren’t HTML specials. It’s specifically those five characters.

Fixed on both branches, both merged:

• dev-branch (1.5.10) — https://github.com/FOGProject/fogproject/pull/1398
• working-1.6 — https://github.com/FOGProject/fogproject/pull/1397

The decode now lives in one place that registration and checkcredentials.php
share, so the two can’t disagree about the same password again, and it no longer
escapes anything. Verified end to end rather than by reading the code: real
POSTs to service/auto.register.php against a real database, on the unfixed
tree and the fixed one. Passw0rd is accepted by both; a password with an &
is rejected by the unfixed tree with your exact “Invalid Login” and accepted by
the fixed one.

Once you’re on a build carrying that, your original & password will work
again. No need to keep the changed one.

Still outstanding — the two MariaDB instances. That’s unrelated to this bug
and I’d still like it sorted, because two database servers on one box is a real
problem waiting to happen:

systemctl list-units 'maria*' 'mysql*'
ss -lntp | grep -E '3306|mysql'

If both are listening, the one FOG is talking to is whichever
/var/www/fog/lib/fog/config.class.php points at — and the other one is holding
a stale copy of your data that will look convincingly real the first time
something connects to it by accident.

On the blank/null task rows from your earlier queries: on your box there
were zero active ones. The blanks are completed and cancelled history for
hosts and images deleted over the years (about 300 hosts and 459 images since
2017), which is expected and harmless. The reaper added in this round only
touches tasks that are still active and can no longer run, so it won’t go near
that history.

[Tom Elliott]

@maxcarpone I would ask you, if you’re daring/willing (it’s considered beta but uses the same pipeline with a lot more modern ui, I need people testing, and Fog_newb can likely attest the new ui look and feel though I think they ran into an issue and needed to get functional right away so the snapshotted back)

Upgrade to working-1.6.

I have a goal (along with @JJ-Fullmer ) to try to get working-1.6 to be master/stable branch by October.

1.6 has been “stagnant” since around 2017 and was in relatively stable grounds back then even.

With AI (as you undoubtly can see I’m using to help drive some things) it’s allowed us to get a lot more coding/refactoring and will hopefully present a much better experience of things on the UI side. Without testing I cannot fix UI bugs though.

AI can do some cool things, but it doesn’t know what “wrong/right” looks like, and JJ and I are only 2 people.

There are others on working-1.6 but more feedback is always good.