Issue joining domain & activating Windows after deployment

[fry_p]

@fry_p said in Issue joining domain & activating Windows after deployment:

If all else fails, please post the contents of your fog.log from a pc that is not automatically rebooting.

Doing this will help our friends diagnose the issue. You can find it either at C:\fog.log or at C:\Program Files (x86)\FOG\fog.log depending on where you chose when you installed the client.

[ckasdf]

Attached log: fog.txt

Totally right, fry_p, I’d forgotten that step. One recurring theme that might point to an issue is this quote: “Response Module is disabled globally on the FOG server.”

I’ve looked around at the FOG web console, and I’m not seeing any buttons, switches, or knobs for Modules, let alone the Response Module.

Looking up that warning, Frank in this thread inferred that he fixed his issue by restarting an agent, but I’m not sure what agent that is.

Critchleyb in this thread mentioned that they migrated hosts from an old server, which I had done; is there a step I may be missing?

[fry_p]

@ckasdf I am seeing some authentication issues in the log. If you did migrate from an old server to a new one, please take a look at this article from the wiki https://wiki.fogproject.org/wiki/index.php?title=FOG_Client#Maintain_Control_Of_Hosts_When_Building_New_Server
I believe you can follow the first set of instructions (copying the ssl directory from the old server to the new one and rerunning the installer). I would strongly advise against recreating the CA (second set of instructions in this part of the article) because it will essentially break all existing fog client installations.
If you do not have access to the old server, I think someone else may have to help as that is beyond my knowledge.

[fry_p]

@ckasdf I realized I was only looking at the beginning of the log and replied before I read enough. So it looks like it is authenticating properly as of 5/9. Let me look for a few moments, but chances are you followed proper protocol when you migrated.

[fry_p]

@ckasdf Please do the following for me. Log into the FOG web gui and search for the host. Select the host and click the “Service settings” tab. Please take a screenshot of this so I may compare to my setup.

Please also, while in the web gui, click on the three gears icon (Client settings) and pick the task reboot tab. Please let us know which boxes are ticked.

Finally, please click on the wrench (FOG Configuration) and click on FOG Settings on the left column. Scroll down to the FOG Client - Task reboot heading and click on it. Please let us know what is checked there.

[ckasdf]

I have attached a screenshot of the Host Module Config page.

Client settings > Task Reboot

• [ √ ] Task Reboot Enabled?
• [ √ ] Task Reboot Enabled as default?
   

Fog configuration > FOG Client - Task Reboot

• [ √ ] CLIENT TASKREBOOT ENABLED
• [    ] TASK FORCE REBOOT

[ckasdf]

Yesterday, I deployed 3 identical laptops with the same image I’ve been working with in this post. Two of them exhibited what I describe, sitting at the local admin login without the ability to sign into the domain. One of them, however, actually worked as expected and allowed me to sign in for the first time using my domain credentials.

On all of them, each time I logged in, a command prompt would briefly pop up, displaying the following:

C:\Windows\SysWOW64>sc config FOGService start= auto
[SC] OpenService FAILED 5:

Access is denied.

It then would proceed to shut down based on the second line of SetupComplete. I had to delete the file from C:\Windows\Setup\Scripts of each computer in order to stop them from shutting down.

[fry_p]

@ckasdf This is very strange you are getting an “Access is denied” because setupcomplete.cmd runs in the local system permission context. This is indeed the issue I think. So in my eyes, the hostname changer in the fog service is running fine, but the service is failing to start with the setupcomplete.cmd script thus causing your issue. The screenshots that I asked for pretty much mirror my setup.

So unfortunately I myself cannot recommend anything but to rebuild the image as it seems the local system account is borked. @Sebastian-Roth @Wayne-Workman @george1421 may have other ideas because they are actual wizards.

[ckasdf]

@fry_p said in Issue joining domain & activating Windows after deployment:

@Sebastian-Roth @Wayne-Workman @george1421

Any of you have thoughts on this? Perhaps the System account is indeed messed up? Any tests I can run for further diagnostics?

[Sebastian Roth]

@ckasdf Sorry for not having engaged in this discussion yet but unfortunately I am way more a wiz when it comes to Linux. I have not toyed with setupcomplete.cmd yet at all. Possibly @george1421 has an idea on why this is making problems?

@fry_p What OS version do you use? Just wondering if this is something that broke with 1809 or 1903 or what?!? @ckasdf Which version do you have?

It’s very interesting it happened to work for one out of three machines!! To me this sounds like it’s very likely to be a timing issue. But on the other hand “Access denied” is a pretty clear statement as well. Possibly the system account is not ready yet or the windows services framework?!?

[fry_p]

@Sebastian-Roth I used 1803 up until last week. We are making new images with 1903 and the behavior is still normal for me. Very strange

[george1421]

@Sebastian-Roth said in Issue joining domain & activating Windows after deployment:

@george1421 has an idea on why this is making problems?

@ckasdf was the golden image created from OEM media or did you use the MS VLK media to create this image?

Was this golden image sysprepp’d?

I find it strange that setupcomplete.cmd is only running when someone logs in. This should not happen (ever) since there should be no connection with setupcomplete.cmd and the login process. This batch file is run by WinSetup at the end of OOBE and just before the first login prompt is displayed.

I could see a case if someone used OEM media where setupcomplete.cmd would not run and was using a first run section of the unattend.xml file to run it where windows would be confused and wait to start the fog service until someone logged in.

That also brings me to the error message about starting the fog server. When the setupcomplete.cmd batch file is run at the end of OOBE, it is executed in the SYSTEM user context. When it was run after login it is run in the context of the current user. Even if the current user is a local admin, it would need to be run from an elevated command window to interact with service settings. So I understand why fog is failing to start when the user logs in.

[ckasdf]

@Sebastian-Roth

Sorry for not having engaged in this discussion yet

Sorry, my turn to have ghosted the conversation. It’s been rather busy and rushed here, but I finally hit another slow spot. I came in with 20 computers to build last week, got those done, and worked on two more this week. No more to do probably for the next couple days.

What OS version do you use?

I have images for both 1803 and 1809, and I’m slowly moving toward 1903. When I posted this question, it seemed that 1803 had more issues with than 1809, but it may have been something wrong with the image itself.

It’s very interesting it happened to work for one out of three machines!!

Indeed. Seemed like a race condition or something, similar to what you mentioned for timing. At this point, I’ve got it consistently joining the domain without needing to use the local admin account (and I didn’t do anything significant to bring that about, oddly enough). However, SetupComplete still runs at every login, and I see that as a potential issue, if only for the confusion it can sometimes cause users to see the command window pop open and run commands (and in at least one or two cases, the window didn’t go away, resulting in helpdesk tickets being submitted).

But on the other hand “Access denied” is a pretty clear statement as well. Possibly the system account is not ready yet or the windows services framework?!?

I’m definitely not sure what to make of it. My guess is when it’s running successfully the first time (while at the login page for the first time), it’s running with System privileges. Once it tries running when the user signs in, it might be trying to run as the user, who doesn’t have permission to modify services.

[ckasdf]

@george1421

Sorry for disappearing, work got extra busy. Some time freed up this week though.

was the golden image created from OEM media or did you use the MS VLK media to create this image?

I believe it was VLK media? My manager who gave me the ISO told me it was a volume license image, which he said I should use instead of the one I had downloaded from Microsoft while testing and developing. Is there a way to look inside the ISO after mounting it to determine its licensing?

Was this golden image sysprepp’d?

It was indeed. Installed Windows, pressed CTRL + SHIFT + F3 at the OOBE setup, did all the configurations within Audit Mode, then ran sysprep /generalize /oobe /shutdown /unattend:c:\unattend.xml. Booted the VM to network, captured to FOG, then deployed from there.

I find it strange that setupcomplete.cmd is only running when someone logs in.

I mentioned this in my reply to Sebastian a few minutes ago, but it seems something I did fixed the primary issue: SetupComplete is now running without needing to log in. However, it ALSO runs at every login, instead of quitting, so there’s still something strange afoot.

This should not happen (ever) since there should be no connection with setupcomplete.cmd and the login process.

That’s definitely my understanding, thus the confusion haha.

I could see a case if someone used OEM media where setupcomplete.cmd would not run and was using a first run section of the unattend.xml file to run it where windows would be confused and wait to start the fog service until someone logged in.

I’m pretty sure it’s volume media that I’m using, but as asked to Sebastian, do you know a way to check the ISO to confirm that it’s VLK media? Also, for reference, linked here is a scrubbed copy of the unattend.xml file (though I had to save it as a *.txt file to be allowed to upload it). One thing I noticed while going through is this:

        <LogonCommands>
            <AsynchronousCommand wcm:action="add">
                <RequiresUserInput>false</RequiresUserInput>
                <CommandLine>C:\Windows\Setup\Scripts\SetupComplete.cmd</CommandLine>
                <Description>Script run upon setup completion</Description>
                <Order>1</Order>
            </AsynchronousCommand>
        </LogonCommands>

I was using the Windows System Image Manager (SIM) feature from the Windows Assessment and Deployment Kit (ADK) to create the Unattend file, but perhaps I did something wrong? I would think SetupComplete being inside the LogonCommands block might be what’s causing my problem?

Hopefully this brings us nearer to resolution.

Edit: fixed formatting involved with the attached/linked file.

[george1421]

@ckasdf said in Issue joining domain & activating Windows after deployment:
sysprep /generalize /oobe /shutdown /unattend:c:\unattend.xml.

I would only place the unattend.xml in the c:\windows\panther directory. That is the first place winsetup looks when it starts, If you define one, as in your case c:\unattend.xml and it finds one in panther first, it will use the panther one over the defined one.

I mentioned this in my reply to Sebastian a few minutes ago, but it seems something I did fixed the primary issue: SetupComplete is now running without needing to log in. However, it ALSO runs at every login, instead of quitting, so there’s still something strange afoot

The execution of setupcomplete.cmd is managed by WinSetup/OOBE. That is the only process that will execute that script. If windows login is executing that script then you have something amiss there.

I’m pretty sure it’s volume media that I’m using, but as asked to Sebastian, do you know a way to check the ISO to confirm that it’s VLK media?

By looking at the cdrom, I don’t think there is a way. I can tell you that oem media will not accept vlk keys and vlk media won’t accept oem keys. Does your boss know if he downloaded the dvd image from the MS VLK site? If you have VLK keys (MAK or KMS) you must have access to the media 2 because they are in the same area on the VLK site.

is a scrubbed copy of the unattend.xml file
Ah, that is why setupcomplete.cmd is running at each login, because you put that in your unattend.xml file. Yeah it doesn’t belong there.
Only WinSetup/OOBE media will call the setupcomplete.cmd.

Just for reference here is a scrubbed version of my unattend.xml file. This is the same one I used for win7 and win10: https://forums.fogproject.org/topic/11920/windows-10-1803-sysprep-problem/7

[ckasdf]

Thanks for the insight! I copied one of my build images to a test-copy, then made two changes. I then ran sysprep /generalize /reboot /unattend and allowed the VM to sysprep and reboot to see what happens. I didn’t capture it to run it through FOG, but I imagine the results will be mirrored once I do.

1. I noticed that your unattend file didn’t contain a reference to SetupComplete. Therefore, I edited unattend.xml to remove the <Component> block that contained <AutoLogon> and <LogonCommands> (within which was contained the reference to SetupComplete.cmd).

2. I moved unattend.xml to the Panther directory at your suggestion. When I ran sysprep, I tested it without defining a file, and sure enough, it picked it up with no problem.

After allowing sysprep to do its job, I logged in as the local admin (since I didn’t have FOG to join the domain for me), and no command windows popped up to try running SetupComplete as the local user. SetupComplete didn’t appear in the Startup tab of Task Manager. FOGService was set to Automatic and was Running. In other words, the things that weren’t supposed to be happening no longer were, and the things that were supposed to happen WERE!

 

By the way, slightly on the topic, I know the wiki suggests restarting the computer with a command in SetupComplete after enabling the FOG Client, but Microsoft’s documentation suggests this is a bad idea:

Warning You cannot reboot the system and resume running SetupComplete.cmd. You should not reboot the system by adding a command such as shutdown -r. This will put the system in a bad state.

My SetupComplete file contains two lines:

sc config FOGService start= auto
net start FOGService

It seems to still restart if needed, so maybe the shutdown line can be removed?

 

Thanks to @fry_p, @george1421, and @Sebastian-Roth for your contributions in helping me figure out this strange issue!