power management scheduled shutdown - flakey

[Wayne Workman]

On fog 1.3 rc-2 with client 0.11.4

I’ve got every host configured to shutdown at 10 pm.

Sometimes I come in for work and my computer is off. Sometimes it’s still on. It’s never asleep, sleep is disabled.

I’ll try to get a log on the morning of when it fails.

[Joe Schmitt]

Issue has been fixed and the patch will be available when 0.11.5 is released.

[Wayne Workman]

Here’s an issue, and is of flaky nature.

I turned on one of our laptops this morning. As soon as it was powered up and at the login screen, it immediately shut down. This computer was shutdown overnight, I shut them down before I left so the batteries could charge better and so they wouldn’t run hot.

The issue occurred this morning, at this time:
7/28/2016 7:30

It appears the logs jump from one date to another. I’m wondering if hibernate and fast startup are disabled, I’m going to guess not. Windows 10 by default does not shut down, when you click shutdown it actually hibernates - but says shutting down (outright lie). Is there anyway to double check the current time and date before the power management task is executed? It seems like if the client misses the exact moment it executes the task later. This is fine, but doublechecking the current time is somewhere close to the scheduled time would be good. I suppose the window should be as large as the client sleep time + maybe an extra two minutes.

Here’s a screen shot of it’s settings and log.

0_1469709652622_fog.log

[Wayne Workman]

I think that if fast startup (sneaky hibernate) is turned on, it would explain this issue as well: https://forums.fogproject.org/topic/8187/fog-client-0-11-2-not-starting-right/9

This is actually the same model laptop with the same image.

[Wayne Workman]

But, what if people want fast startup turned on? It really does speed up starting up a computer in the mornings, it has real benefit. For imaging purposes, yeah I think it should be turned off before capture because it’s caused me nothing but problems there. However, post-deployment group policies may turn it on. A snapin could even turn it back on.

Now, what happens when every computer is turned on in the morning via WOL, and then immediately shuts down by themselves? What if a teacher turns on a computer, and then it immediately shuts down. That’s not acceptable. I think simply verifying the current date/time just before power management tasks would solve all this.

@Joe-Schmitt

[Tom Elliott]

@Wayne-Workman I could be wrong, but power management “flakey” ness sounds, to me, like it simply hadn’t run. So the time it new it had to run never ran, and because the next time it turned on it was “after” that point in time it ran properly.

@Joe-Schmitt If I’m wrong I’m wrong, just let me know.

I say what I say because this same principle is how the FOGScheduler service works.

It doesn’t know about the “next” run time until the original time is met.

So, for example, if 0 22 * * * crontab failed for one reason or other, the time would not change from when it was expected… So the next time it does run, the prior tab is still in place and the conditions are met. Once it runs, the conditions are already met and it set’s the next run time.

[Wayne Workman]

@Tom-Elliott said in power management scheduled shutdown - flakey:

I could be wrong, but power management “flakey” ness sounds, to me, like it simply hadn’t run. So the time it new it had to run never ran, and because the next time it turned on it was “after” that point in time it ran properly.

I follow what you’re saying. But re-checking the current date and time just before a power related task would completely solve the issue. It’s not acceptable for someone to turn on a computer, only for it to immediately turn itself off. Additionally, it’s not a server issue. It’s a client issue.

[Tom Elliott]

@Wayne-Workman It’s not about checking the date and time. The date and time is what determines it.

So if the task was scheduled to run at 7/27/2016 at 2200 and the device was shut off at 7/27/2016 at 1800 (because the checks see if the time is in the past) You could wait 6 months before turning it on again, and it had no chance to update.

[Wayne Workman]

@Tom-Elliott It doesn’t need to update. It just needs to check the current date and time just before the task.

if the task was to shut down at 10 pm, and the computer tries to do that at 8 AM, how much sense does that make? None. If it simply checks the local system time, It’d know that the time was missed by too much and to skip that power task.

[Tom Elliott]

@Wayne-Workman That’s just it. The check is to see if the current time is past the time that was set.

That’s how cron’s work. You can give it a “period” of adjustment I suppose, but it isn’t going to know the difference because it’s based on the task time itself in whole.

For example. 7/27/2016 - 2200 < 7/27/2016 - 2201 (therefore it knows it has to do the action and update the next time it should run). Same goes for 7/27/2016 - 2200 and January 15, 2019 - 0000. It has to do the check to know whether to run it in the first place.

Remember, we’re not dealing with linux for the crontabs, rather services that act similar (but are not equivalent) to real crontabs.

So a grace period may be needed, but what’s a suitable one? For a minute task I suppose it doesn’t matter, but for an hourly task should you wait an hour, 5 minutes, 10 minutes, 14 seconds? You understand?

[Wayne Workman]

@Tom-Elliott I understand how it currently works, Tom. And how it’s currently working isn’t acceptable.

[Tom Elliott]

@Wayne-Workman Isn’t acceptable by who?

[Wayne Workman]

@Tom-Elliott By anyone who turns on a computer only to have it immediately shut down.

I can’t use it as it is.

[Tom Elliott]

@Wayne-Workman If you’re running power management, why not leave your systems running and let powermanagement handle it?

[Tom Elliott]

@Wayne-Workman

I think the “simplest” solution (if this is not already the case) for power management is to simply keep the management tasks in memory. So they are stored in such a way that if you shutdown the fog service, it has to recreate the new date/time. Dealing with math of timestamps would work, but is imperfect due to the nature of how the crontasks are operated. Simply by having it recheck the next run time when the service restarts is much nicer and would be completely independent to the systems.

[Wayne Workman]

@Tom-Elliott The service isn’t shutdown when a system hibernates, which is exactly how windows fast startup operates. The local host’s time needs checked every time the fog client thinks a power task should be ran - and if the local time and the task’s scheduled time match within a given window, then run the task. something like:
(Local time <= scheduled time + sleep time + 2 minutes)

[Tom Elliott]

Three approaches.

The 1st and theoretically the most simple. On service shutdown/restart (before it has stopped, but while in the processes of stopping) Remove and scheduled tasks created by the module. (which means we need a way to find out which task scheduler tasks were created by PM to begin with).

The 2nd and most simple, but could pose problem on next boot: On service start, clear out any scheduled tasks created by the service (which means we need a way to find out which task scheduler tasks were created by PM to begin with).

The 3rd, is only make powermanagement handled by the service itself. Fairly difficult to implement easily, but once done fairly fail proof. This way when unloading the service, the tasks are auto magically cleaned up (and we don’t have to manage different elements of the system as it would only care about what it’s received).

[Tom Elliott]

@Wayne-Workman Because it’s cron, you can only do a scheduled time + 2 minutes, but even still. If the system is hibernating for a “shutdown” how did you create the image?

[Wayne Workman]

@Tom-Elliott said in power management scheduled shutdown - flakey:

If the system is hibernating for a “shutdown” how did you create the image?

Good question. No idea.

[Tom Elliott]

Oh, and to add on. If you based it on the sleep time – yes I know you can change it --, may run into MANY more strange things. For Powermanagment, the checks should be every minute, regardless of Cycle checktime.

[Wayne Workman]

@Tom-Elliott I can agree on that.