Huge database entries number

[siarkowski]

@Tom-Elliott
Thanks for replay.

In the tasks menu, scheduled Tasks there are no entries. However but in database table scheduledTasks the are two entries:

±-----±------------±-------±-------±-------------±---------±-------±------±--------±------±----------±--------------±----------±-----------±---------±---------±---------±---------±---------±-----------±---------+
| stID | stName | stDesc | stType | stTaskTypeID | stMinute | stHour | stDOM | stMonth | stDOW | stIsGroup | stGroupHostID | stImageID | stShutDown | stOther1 | stOther2 | stOther3 | stOther4 | stOther5 | stDateTime | stActive |
±-----±------------±-------±-------±-------------±---------±-------±------±--------±------±----------±--------------±----------±-----------±---------±---------±---------±---------±---------±-----------±---------+
| 1 | Deploy Task | | S | 1 | | | | | | 1 | 6 | 0 | | | -1 | fog | 1 | | 1681921800 | 0 |
| 2 | Deploy Task | | S | 1 | | | | | | 1 | 8 | 0 | | | -1 | fog | 1 | | 1741105800 | 0 |
±-----±------------±-------±-------±-------------±---------±-------±------±--------±------±----------±--------------±----------±-----------±---------±---------±---------±---------±---------±-----------±---------+

[Tom Elliott]

@siarkowski Thanks, the scheduledTasks you’re showing aren’t showing anything out of the ordinary other than these were “Delayed” tasks

First was delayed to run at 2023-04-19 16:30:00 (UTC)
Second was delayed to run at 2025-03-04 16:30:00 (UTC)

I am pretty sure, this isn’t the issue you’re seeing at least.

So I wonder if there’s some corruption due to running out of disk space that’s causing this?

[siarkowski]

@Tom-Elliott
I finished all running deploy tasks, cleaned task table.
At this moment partition was over 5GiB free disk space.
I started 37PCs deploy task and the story is same again. Database tasks table files were 40kB before I started the task and after 30 minutes database tasks table files are over 4MB.
Services work fine, tasks are processed, but tasks table is over 26000 records long and it’s increasing.
I’m pretty sure if I start task for 200 PCs disk will run out of space after few hours

[Tom Elliott]

@siarkowski I understand what you’re saying, what I don’t understand is why this is happening at all. If this was a normal issue, we’d have been notified of this issue probably many many times.

I’m not saying this couldn’t be a bug, I just have no way to replicate the issue in a consistent manner.

Might you be able to pull a backup of your database using FOG Configuration -> Configuration Save -> Export

(After cleaning up the Task issue) and see if you can use a secondary system to import that DB and (as a test) and ensure all the data you require is there (not necessarily the physical images)).

I don’t know what to suggest or say here other than I suspect the database is corrupted and it’s that corruption adding and exploding task records. This (again) isn’t to say there isn’t a bug, but I want try to start with a clean baseline and see if things worked properly under “sanitized” environment.

If a fresh server (if you can share your images that much better) works without adding/exploding the task table, that at least proves it’s a DB Corruption issue. If it, too, starts exploding in task table records, then it shows there’s something in the code that’s doing this unexpectedly but at least we narrowed the scope to exactly where to being looking (somewhere in the task checkin process it seems?)

[siarkowski]

@siarkowski
Corrupeted DB or schema is highly possible.
I’ll backup tables: users, hosts, groups and images data and make a fresh FOG install.
I"ll let you know as soon as I’ll do it. It will be no sooner then next week I’m afraid.

[Tom Elliott]

@siarkowski That’s perfectly understandable and much appreciated. Sorry I don’t have a simple “here’s what to do” right now, and maybe a fresh install will fix the issue altogether, but right now it we have limited information. Thank you again for your understanding a patience with this as well.

[siarkowski]

@Tom-Elliott
As a curiosity, after two hours of 37 PCs deployment , 10 are finished, 10 are rolling, 17 are waiting. Tasks table is 815000 records long and tasks db files are 138MiB total.
Don’t get me wrong, FOG is awesome tool - seriously.

[siarkowski]

@siarkowski

Hello All,

First hit.
I uninstalled FOG, dropped database, installed FOG again, imported users, hosts, images, started task for 31 PCs and records with taskimageID equal to 0 are being inserted again.

Second hit.
I imaged OS disk, installed fresh operating system, installed FOG, imported users, hosts, images, started task for 31 PCs and records with taskimageID equal to 0 are being inserted again.

Third hit.
I reverted all changes, installed FOG but rev. 1.5.10.1622, imported users, hosts, images, started task for 31 PCs and no records with taskimageID equal to 0 are being inserted Not to be so happy another issue appeared, tasks do not start automatically I have to force it to start, but it’s different story.

[JJ Fullmer]

@siarkowski I believe I have found the cause of this.
A while back, right after the version you reverted too, we added an improved queueing system which is working perfectly in 1.6. However when we ported it backwards into 1.5.10.x I made a simple syntax error (the wrong $task->id vs $task->get('id') ). I have fixed this in the latest dev-branch.

This should also greatly improve the experience of the imaging task queue (see also https://github.com/FOGProject/fogproject/issues/736 and https://github.com/FOGProject/fogproject/issues/691) I thought I also wrote a post somewhere in the forum walking through the updated process that fixed some longstanding date math issues, but I can’t find that now.

Point being, if you would be so kind as to update to the latest dev-branch version and see if it fixes the issue, that would be very helpful.

[siarkowski]

After upgrading to 1.5.10.1754 it works just fine.
Thanks for bug tracking and improvement!