power management scheduled shutdown - flakey

[Wayne Workman]

On fog 1.3 rc-2 with client 0.11.4

I’ve got every host configured to shutdown at 10 pm.

Sometimes I come in for work and my computer is off. Sometimes it’s still on. It’s never asleep, sleep is disabled.

I’ll try to get a log on the morning of when it fails.

[Wayne Workman]

@Joe-Schmitt Does this change only apply to scheduled shutdown tasks, or to all power management tasks including reboot? Just asking.

[Joe Schmitt]

Issue has been fixed and the patch will be available when 0.11.5 is released.

[Joe Schmitt]

Well, time to reply to this massive notification spamming thread.

This is not how PowerManagement works. I leverage the Quartz.Net library. I simply provide a Quartz formatted CRON schedule and it takes care of the rest. Seeing as how I have not scoured through their source code I can only speculate what is going on.

With the sudden time change, quartz realizes that an event was missed, or what’s known as a misfire. This allows some redundancy, e.g. if a job is set to shutdown at 22:00, and the event fires at 22:01 it will still be processed.

I believe the current behavior is configured to execute a misfire regardless of how much the time is really off. I will work on this and push out a 0.11.5 when I have a chance.

[Wayne Workman]

@Tom-Elliott I can agree on that.

[Tom Elliott]

Oh, and to add on. If you based it on the sleep time – yes I know you can change it --, may run into MANY more strange things. For Powermanagment, the checks should be every minute, regardless of Cycle checktime.

[Wayne Workman]

@Tom-Elliott said in power management scheduled shutdown - flakey:

If the system is hibernating for a “shutdown” how did you create the image?

Good question. No idea.

[Tom Elliott]

@Wayne-Workman Because it’s cron, you can only do a scheduled time + 2 minutes, but even still. If the system is hibernating for a “shutdown” how did you create the image?

[Tom Elliott]

Three approaches.

The 1st and theoretically the most simple. On service shutdown/restart (before it has stopped, but while in the processes of stopping) Remove and scheduled tasks created by the module. (which means we need a way to find out which task scheduler tasks were created by PM to begin with).

The 2nd and most simple, but could pose problem on next boot: On service start, clear out any scheduled tasks created by the service (which means we need a way to find out which task scheduler tasks were created by PM to begin with).

The 3rd, is only make powermanagement handled by the service itself. Fairly difficult to implement easily, but once done fairly fail proof. This way when unloading the service, the tasks are auto magically cleaned up (and we don’t have to manage different elements of the system as it would only care about what it’s received).

[Wayne Workman]

@Tom-Elliott The service isn’t shutdown when a system hibernates, which is exactly how windows fast startup operates. The local host’s time needs checked every time the fog client thinks a power task should be ran - and if the local time and the task’s scheduled time match within a given window, then run the task. something like:
(Local time <= scheduled time + sleep time + 2 minutes)

[Tom Elliott]

@Wayne-Workman

I think the “simplest” solution (if this is not already the case) for power management is to simply keep the management tasks in memory. So they are stored in such a way that if you shutdown the fog service, it has to recreate the new date/time. Dealing with math of timestamps would work, but is imperfect due to the nature of how the crontasks are operated. Simply by having it recheck the next run time when the service restarts is much nicer and would be completely independent to the systems.

[Tom Elliott]

@Wayne-Workman If you’re running power management, why not leave your systems running and let powermanagement handle it?

[Wayne Workman]

@Tom-Elliott By anyone who turns on a computer only to have it immediately shut down.

I can’t use it as it is.

[Tom Elliott]

@Wayne-Workman Isn’t acceptable by who?

[Wayne Workman]

@Tom-Elliott I understand how it currently works, Tom. And how it’s currently working isn’t acceptable.

[Tom Elliott]

@Wayne-Workman That’s just it. The check is to see if the current time is past the time that was set.

That’s how cron’s work. You can give it a “period” of adjustment I suppose, but it isn’t going to know the difference because it’s based on the task time itself in whole.

For example. 7/27/2016 - 2200 < 7/27/2016 - 2201 (therefore it knows it has to do the action and update the next time it should run). Same goes for 7/27/2016 - 2200 and January 15, 2019 - 0000. It has to do the check to know whether to run it in the first place.

Remember, we’re not dealing with linux for the crontabs, rather services that act similar (but are not equivalent) to real crontabs.

So a grace period may be needed, but what’s a suitable one? For a minute task I suppose it doesn’t matter, but for an hourly task should you wait an hour, 5 minutes, 10 minutes, 14 seconds? You understand?

[Wayne Workman]

@Tom-Elliott It doesn’t need to update. It just needs to check the current date and time just before the task.

if the task was to shut down at 10 pm, and the computer tries to do that at 8 AM, how much sense does that make? None. If it simply checks the local system time, It’d know that the time was missed by too much and to skip that power task.

[Tom Elliott]

@Wayne-Workman It’s not about checking the date and time. The date and time is what determines it.

So if the task was scheduled to run at 7/27/2016 at 2200 and the device was shut off at 7/27/2016 at 1800 (because the checks see if the time is in the past) You could wait 6 months before turning it on again, and it had no chance to update.

[Wayne Workman]

@Tom-Elliott said in power management scheduled shutdown - flakey:

I could be wrong, but power management “flakey” ness sounds, to me, like it simply hadn’t run. So the time it new it had to run never ran, and because the next time it turned on it was “after” that point in time it ran properly.

I follow what you’re saying. But re-checking the current date and time just before a power related task would completely solve the issue. It’s not acceptable for someone to turn on a computer, only for it to immediately turn itself off. Additionally, it’s not a server issue. It’s a client issue.

[Tom Elliott]

@Wayne-Workman I could be wrong, but power management “flakey” ness sounds, to me, like it simply hadn’t run. So the time it new it had to run never ran, and because the next time it turned on it was “after” that point in time it ran properly.

@Joe-Schmitt If I’m wrong I’m wrong, just let me know.

I say what I say because this same principle is how the FOGScheduler service works.

It doesn’t know about the “next” run time until the original time is met.

So, for example, if 0 22 * * * crontab failed for one reason or other, the time would not change from when it was expected… So the next time it does run, the prior tab is still in place and the conditions are met. Once it runs, the conditions are already met and it set’s the next run time.