FOG service quits running
-
- HOST OS - Windows 10
- FOG Version - 1.5.7.1
- FOG OS - CentOS 7.6
I have noticed on several hosts that the FOG Service quits running after several days. I checked the Windows Services and it shows it set to automatic (or automatic delayed) but not started and it had been off for days.
Here is a snippet of the FOG.log file from one of the hosts. The rest in this group of host looked similar.
------------------------------------------------------------------------------ ----------------------------------UserTracker--------------------------------- ------------------------------------------------------------------------------ 8/20/2019 5:40 PM Client-Info Client Version: 0.11.16 8/20/2019 5:40 PM Client-Info Client OS: Windows 8/20/2019 5:40 PM Client-Info Server Version: 1.5.7.1 8/20/2019 5:40 PM Middleware::Response Success ------------------------------------------------------------------------------ 8/20/2019 5:40 PM Service Sleeping for 311 seconds 9/4/2019 9:13 AM Main Overriding exception handling 9/4/2019 9:13 AM Main Bootstrapping Zazzles 9/4/2019 9:13 AM Controller Initialize 9/4/2019 9:13 AM Controller Start 9/4/2019 9:13 AM Service Starting service 9/4/2019 9:13 AM Bus Became bus server 9/4/2019 9:13 AM Bus Emmiting message on channel: Status 9/4/2019 9:13 AM Service Invoking early JIT compilation on needed binaries ------------------------------------------------------------------------------Is there anyway to enforce that the FOG Service is started and running?
Thanks!
Cheers,
Joe Gill
-
@Joe-Gill Oh wow! Thanks for posting this here!! I had that reported via mail by someone else but have not been able to replicate this issue myself so far. So I am just missing information on why that might happen.
Anything you can find in Windows event logs? I am fairly sure you should find some notice/warning/error right at that point in time what logging stopped.
What Windows 10 version do you have exactly? 1803, 1809, 1903?
-
I will check it out for you. I hadn’t peered into the Windows logs yet. I believe all of the machines with this issue are Windows 10 Ver 1903. But I will get more details here for you directly. In fact I have an entire lab I haven’t manually started the FOG service yet on. I will go check this out now and see what I come up with. If I can’t find anything obvious, you can certainly set up a time to remote into one of the trouble PCs.
Also, I had noticed this randomly happen last year in other labs. It was so random though I figured something else happened or I had done something that caused the issue. This is too wide spread not to be me. LOL!
Thanks!
-
Alright, I have a bit of info to share here…
I have a machine with Windows 10 1709 OS Build 16299.15 and an identical machine (sitting next to it) that has Windows 10 1703 Build 15063.0. The 1709 machine FOG service is not running. The 1703 FOG service IS running.
I checked multiple machines and none seem to have logs that go back far enough to any events that happened on the day the service quit. That said the things these all have in common (on some machines) occurred two days prior to a system update. (8/20/19) The update occurred on 8/22/19… I had a few outliers that happened mid July.
Oddly enough, I was out of the office the week (8/19/19) this event occurred. I checked with the other tech and we had no physical issues that day and the folks that were here used this very lab that day.
So far all of my logs are not showing anything back when the problem occurred. I have noticed that a commonality in the FOG logs is that this issue happens after FOG sleeps or reboots.
Let me know what else I can provide you. I have several machines in the same lab that have this issue. I had a few that after I logged into AD, the service checked in and everything seems to be working.
Cheers,
Joe
-
@Joe-Gill said in FOG service quits running:
I checked multiple machines and none seem to have logs that go back far enough to any events that happened on the day the service quit. That said the things these all have in common (on some machines) occurred two days prior to a system update. (8/20/19) The update occurred on 8/22/19… I had a few outliers that happened mid July.
Thanks heaps for looking into it. Do I get this right, most of the machines having the issue showed (in the logs) the service stopped working on the very same day, all on 8/20/19??
So far all of my logs are not showing anything back when the problem occurred.
So the event logs go back to that date but there is no notice/warning/error in the event logs?
I have noticed that a commonality in the FOG logs is that this issue happens after FOG sleeps or reboots.
Do you mean PC reboot? Can you post an example log when it seems to happen on “reboot”?
-
@Sebastian-Roth said in FOG service quits running:
Thanks heaps for looking into it. Do I get this right, most of the machines having the issue showed (in the logs) the service stopped working on the very same day, all on 8/20/19??
Not a problem! I am happy to help out the project. The service quit on a handful of machines on 8/20/19 and some were in July.
So far all of my logs are not showing anything back when the problem occurred.
So the event logs go back to that date but there is no notice/warning/error in the event logs?
The event logs don’t go back far enough to show anything. In other words the event logs don’t start until after the service quit working (days later).
I have noticed that a commonality in the FOG logs is that this issue happens after FOG sleeps or reboots.
Do you mean PC reboot? Can you post an example log when it seems to happen on “reboot”?
I’ll go grab an entire log and post it here.
-
Here is the entire FOG log on one of the 8/20/19 machines. I can post it as code too if you’d like it. I just thought a log file may be simpler.
Thanks!
-
@Joe-Gill This almost seems like a “Patch Tuesday” kind of issue. Especially considering a large amount of patching was performed just on August 13th. Maybe it had to shutdown some of the services that FOG Client uses during the update process? I’m guessing a simple restart of the machine would have fixed the client, or even manually starting it.
Just thinking, and I could be totally wrong.
-
I just noticed something… While looking for log files again this morning… I had one machine that the service literally quit working yesterday… My event logs show events just AFTER the FOG service quit logging…
This machine is on Windows 10 1709 build 16299.15…
Guess what happened after that? Windows ran an update!
We do all of our updates from a Windows WSUS…
Hopefully this helps!!
-
I will try a reboot on the machine I had quit checking in yesterday and see.
-
So I rebooted two different machines that were having this problem. The service restarted on one and not the other. Ha!
Thanks!
-
@Joe-Gill said in FOG service quits running:
So I rebooted two different machines that were having this problem. The service restarted on one and not the other. Ha!
Thanks!
I checked the update settings on the machine that the service wasn’t running after the reboot… You guessed it, a Windows Update was running (installing). I’m gussing that is the caust of this entire issue.
Reboot will likely resolve it after the update install is completed. Hope this helps someone else down the road.
-
Gotta love windows updates sometimes.
-
@Joe-Gill Nice catch!! Sounds very much reasonable. Will be interesting to see if we are actually able to replicate the issue on demand and see at which state exactly it kills the fog-client. Probably a big project for Winter I reckon…
-
Let me know how I can help. I have a lab that gets used very infrequently that I can test things on if need be. (Time permitting…) My time begins to get more free here in about a month or two and then gets more busy again in February. I’d be more than happy to help out the project!
-
Try configuring the servicio with this:
sc.exe config FOGService start=delayed-auto sc.exe failure FOGService actions= restart/60000/restart/60000/restart/ reset= 120.It work’s to save me from stopping Fogservice errors.
-
@Joe-Gill Are you still around? Would be great to hear if you still see this issue on some of your machines. I have tried to replicate this problem but wasn’t able to. See here: https://forums.fogproject.org/topic/13902/fog-service-and-interventions
