Hostname Changer AD Issues
-
Hello,
I wanted to post this in general as I am unsure of the correct area to post issues with Hostname Changer.
Currently, we just started experiencing issues with Hostname Changer over the past couple of weeks. There has been no maintenance done on the Fog Server, No image changes, and no changes on the back end previous to the issues we started to experience.
We are running FOG Version 1.5.9 with Debian 10. We have also tried a couple of the trunk versions as well for troubleshooting this issue, including 1.5.9.60. We also tested with the latest Kernel build. None of these seem to have resolved the issues we are experiencing, nor did they have any change in the issue we are experiencing.
The issue that we have started to see somewhat broad, but I will explain as best I can.
-
On new images, if the PC doesn’t exist Hostname Changer will no longer create an AD Object for the PC.
-
Sometimes imaging a computer using the name of a pre-existing AD object will cause additional trust relationship issues, but FOG will act like it’s on the domain.
Ex: We had a computer named TEST that we are replacing with a new device. The device images and successfully renames to TEST. Initially we are able to access the computer with a domain account, but after a few moments it will receive a trust relationship error. FOG.log will show the device is joined to the domain with code 2691. Using Control Panel, we can rename the computer to TEST-01 and put it into a workgroup. FOG will then rename the device back to TEST and add it to the domain, but we’ll receive the same error. We can rename the computer in FOG to TEST-01 and it’ll add it in AD as a new object. We can then delete the old TEST object and rename it again using FOG. That will typically allow the computer to work correctly under the name TEST.
-
Renaming a computer does not create a new AD object. However, it is still able to access the domain (no sign issues or trust relationship errors). Adding a new computer with the previous name will cause trust relationship issues on the new computer.
-
Sometimes renaming a PC WILL create a new AD object, which is different from number 3 above.
These are just some of the scenarios that we are experiencing, I can list them all but there are a vast number of scenarios that occur.
One thing I did find on an old forum post, is the following setting:
Fog Configuration > General Settings > CHANGE HOSTNAME EARLY
Turning this off allowed new AD objects to be created successfully on new images with no preexisting AD entry. But a plethora of other issues still exist with Hostname Changer that didn’t exist before.
The strange thing is, this setting has been on and in place for years without issue.
We have also verified the domain account being utilized is active and has all permissions and attributes necessary to successfully add PCs to the domain. We tested this manually with a few PCs with 100% success.
Any thoughts or suggestions on where to look or troubleshoot would be appreciated!
The Fog Log shows the PC as on the domain in most of these cases with no errors.
Thanks!
-Lambo
-
-
Currently, we just started experiencing issues with Hostname Changer over the past couple of weeks.
Can you be more specific on when this started? Did it work without issues before? Just wondering if it could be Windows or .NET updates or something else that brought this issue up.
We are running FOG Version 1.5.9 with Debian 10. We have also tried a couple of the trunk versions as well for troubleshooting this issue, including 1.5.9.60. We also tested with the latest Kernel build.
None of those things would play a role in this case. Not the kernel used and there have not been changes in how FOG handles hostname changes between 1.5.9 and 1.5.9.60. As well, the fog-client being involved a lot has not been updated.
Did you update from an earlier version to 1.5.9 when this started to happen?
The issue that we have started to see somewhat broad, but I will explain as best I can.
On new images, if the PC doesn’t exist Hostname Changer will no longer create an AD Object for the PC.
If the fog-client version used has not changed I can only see other components (Windows build version, Windows Updates, .NET Updates, AD server Updates, GPO, …) causing this.
Don’t get me wrong. I am not saying this is non of our business. What I am asking for is a detailed description of components that changed since it worked last. Otherwise this won’t get us anywhere I suppose.
- Which Windows 10 build version do you use?
- Which AD server version?
- All current updates installed on the systems?
One thing I did find on an old forum post, is the following setting:
Fog Configuration > General Settings > CHANGE HOSTNAME EARLYJust to clarify what this option really does is that it edits the Windows registry files after deploying the image to disk (before the very first reboot!) to change the computer name. This has worked with Windows 7 in the past and I have tested this with Windows 10 not long ago. Though I did not test this in a setup with AD integration. So I can’t say if this would cause any side effects.
Turning this off allowed new AD objects to be created successfully on new images with no preexisting AD entry.
That’s an interesting point! From my point of view there is no need to have CHANGE HOSTNAME EARLY enabled if you use the fog-client doing the renaming for you. As you say it shows less issues, just leave that switched off.
-
Thank you so much for your reply, I have seen your name dozens of times while researching various issues. It is a pleasure to speak with you.
Can you be more specific on when this started? Did it work without issues before? Just wondering if it could be Windows or .NET updates or something else that brought this issue up.
This issue was discovered on 1/28/2021 in our organization. We had new PCs to deploy and some technicians discovered the issue, but I am unsure how long it would have existed before we discovered the issue. AD Join / Hostname Changer did work perfectly fine before this issue was discovered.
It is quite possible that this issue is because of a newer .NET issue or similar, it is hard to say with all the changes MS is implementing! We did have a new image created, Version 20H2, but our old image is experiencing issues as well, Version 2004. I can troubleshoot whatever you need though!
None of those things would play a role in this case. Not the kernel used and there have not been changes in how FOG handles hostname changes between 1.5.9 and 1.5.9.60. As well, the fog-client being involved a lot has not been updated.
Did you update from an earlier version to 1.5.9 when this started to happen?
I didn’t think these would play a role in Hostname Changer, but I figured I would give it a try just in the off chance that there was a difference. We upgraded from v1.5.8 to v1.5.9. From my memory, we didn’t have any issues after the upgrade. But see if anyone else noticed anything.
If the fog-client version used has not changed I can only see other components (Windows build version, Windows Updates, .NET Updates, AD server Updates, GPO, …) causing this.
Don’t get me wrong. I am not saying this is non of our business. What I am asking for is a detailed description of components that changed since it worked last. Otherwise this won’t get us anywhere I suppose.
I can certainly see how that could affect the system as well. Give me some time to try and curate a list of these changes, although the issue has appeared on our known good image as well. But I can still compile a list.
Which Windows 10 build version do you use?
Previously we were utilizing 2004, then we moved to 20H2. We always keep one known good version in the event there are issue with the new image.
Which AD server version?
AD Server version is Server 2012 R2
All current updates installed on the systems?
At the time of creation all updates were installed on the image, but i can update a PC after imaging or update the image if you’d like.
Just to clarify what this option really does is that it edits the Windows registry files after deploying the image to disk (before the very first reboot!) to change the computer name. This has worked with Windows 7 in the past and I have tested this with Windows 10 not long ago. Though I did not test this in a setup with AD integration. So I can’t say if this would cause any side effects.
That’s an interesting point! From my point of view there is no need to have CHANGE HOSTNAME EARLY enabled if you use the fog-client doing the renaming for you. As you say it shows less issues, just leave that switched off.
Thank you so much for the information on this setting! I wasn’t sure exactly how it functioned. Unfortunately for me, today it made me a liar, in my testing it isn’t wanting to add the PC to AD even with this setting unchecked. We are just experiencing very very strange issues. It also acts like the PC is on the domain, I can log in with my network account, but the PC is nowhere on the domain and will eventually encounter a Trust Relationship issue.
Let me know if there is anything at all you would be interested in seeing. I can recreate the images, grab logs, test whatever you may need.
One thought I had, is it possible to downgrade from version 1.5.9 to 1.5.8 for testing purposes?
Thanks!
-Lambo
-
@lambo said in Hostname Changer AD Issues:
Give me some time to try and curate a list of these changes, although the issue has appeared on our known good image as well.
Well that’s definitely an interesting point. So do I get this right? That known good image was captured some time before the issue started, no MS updates applied to it, sure. Can you do a series of tests with that image (maybe you’ve done the test already and can report right away)? Deploy it to the exact same host 10 times. Beforehand and in between remove the machine account from the AD. I know this will only look into one of the issues but we need to try and focus on one thing at a time in hope to spot what’s going in. So let it do the deploy, rename and join. Then check AD, join state in the client, fog-client log and user login (does it loose trust to AD?). Note down results for each round.
What I hope to get from this is, if we get persistent behavior with one defined scenario or if we see different outcomes.
-
Hi Sebastian,
Sorry for the delay, we have been quite busy at work.
You are exactly correct, the known good image has the same issue as well. No MS updates on it.
I will certainly test this out! I think this is a great idea and hopefully it produces something tangible we can utilize.
I will report back once finished!
Thanks!
Lambo
-
@lambo Don’t forget to Reset Encryption Data in the FOG web UI for this client so it will checkin straight away every time as well.
Looking forward to see what we get from these tests.
As well you might try taking a look at the event logs of the PC and maybe the AD server as well when you see it doesn’t work as expected.
-
Thank you for that tip! I will ensure this gets done as well.
It may take me some time to get this data to you with our hybrid schedule, but I will be sure to upload it!
I am very interested to see what we get as well. I will also comb through these logs to ensure that there isn’t anything apparent in that area as well.
Thanks!
Lambo
-
Hi Sebastian,
Just wanted to update you that I am combing through the data that I have on the imaging sequences that we spoke about last.
I will be able to upload the process and data to you shortly!
Thanks!
Lambo
-
Ok i Have the data available!
Would you like me to put the summary and all 10 Fog logs right here or is there a different way you prefer I upload these files?
Thanks!
Lambo
-
@lambo I am not exactly sure what size and file types are allowed in the forums. You can give it a try or upload to a file share/cloud of your choice and post a link here.
-
This post is deleted! -
Here is a link you can try, Let me know if you are able to access the drive.
https://drive.google.com/drive/folders/1aOwYRK71pVZL68uIrzPzT1ylOmquVzeP?usp=sharing
Fog logs are listed, Testing Notes.txt are the overall notes showing what tasks have been taken for each fog log.
Let me know if you need anything else!
Thanks!
Lambo
-
@lambo Well done. Looking through your notes and logs I find it very awkward that we have different results for tests that should end up the same! The fog-client simply calls official MS API to do rename, AD join/leave and so on. While it’s still possible that it’s something we do wrong in the code I find it very strange that it would work one time and fail the next few tries.
What I found in the fog-client logs is a few unknown return codes:
FOG Test 1.log: 2/25/2021 9:10:45 AM HostnameChanger Unknown Return Code: 1791 FOG Test 5.log: 2/26/2021 10:19:55 AM HostnameChanger Unknown Return Code: 1791 FOG Test 7.log: 2/26/2021 1:34:39 PM HostnameChanger Unknown Return Code: 1791 FOG Test 8.log: 2/26/2021 2:26:16 PM HostnameChanger Unknown Return Code: 1332 FOG Test 8.log: 2/26/2021 2:28:48 PM HostnameChanger Unknown Return Code: 1332 FOG Test 9.log: 2/26/2021 3:49:57 PM HostnameChanger Unknown Return Code: 1791- 1791: A remote procedure call is already in progress for this thread. (reference)
- 1332: No mapping between account names and security IDs was done. (reference)
Possibly you have some other software interfering with the fog-client? Do you have other tools installed that would run straight after the client comes up after imaging?
Did you find anything obvious in the event logs on the client or the AD server when it fails to create the machine account?
As a side note: You still have the 0.11.19 fog-client installed. Unfortunately the CA certificate shipped with that binary ran out last year and so the auto updater can’t pull up to the 0.12.0 version coming with FOG 1.5.9. It downloads the SmartInstaller but cannot verify the signature and therefore cannot install it. So at some point you’ll need to update your base image with the newer fog-client.
Though I don’t think 0.12.0 will make much of a change in this scenario you are facing here. There have not been updates to the HostnameChanger module since 0.11.19.
-
Sorry for the delay, we have a major project that I am currently involved with in my organization.
I agree! But there must be a cause whether its on our end or something that may have gotten corrupted on our server. This whole issue is quite strange and is very perplexing. I can’t seem to find any rhyme or reason.
I will have a look at these error codes and see if I can’t find anything that would be causing this issue.
The only thing that we have is a snapin for our software deployment software that is set to run after the PC is imaged and on the domain. I can certainly try to run some images without this snapin enabled and see if that makes a difference?
Other than that, i am not seeing anything that could cause issues like we are seeing. I’m not seeing too much in the event logs and I don’t have access to the AD server, but i had the team look at it and they said they haven’t seen anything unusual.
Let me take a look at the Fog client as well, even though you might not think it could be the issue, we can try to ensure that it isn’t causing some funky issue.
Do you suggest maybe downgrading the server version to 1.5.8 as well just to test and ensure we didn’t encounter some weird issue when upgrading, perhaps something got corrupted on our end or similar?
Thanks, sorry again for the delay!
Lambo
-
@lambo said in Hostname Changer AD Issues:
Do you suggest maybe downgrading the server version to 1.5.8 as well just to test and ensure we didn’t encounter some weird issue when upgrading, perhaps something got corrupted on our end or similar?
Don’t think a downgrade will be of help in this case. Also be aware that going back is not something we officially support within the FOG software. It’s hard enough to make updating work for as many people as possible without too much trouble. So it’s a manual process of taking an old DB backup and so on. As well there is no auto-downgrade to an older version of the fog-client.
-
Ok, that is understandable. Is there anything that can verify the installation?
I will work on the Fog client and seeing if upgrading changes anything, as well as imaging without the snapin enabled.
Do you have any other suggestions?
Thanks!
Lambo
-
@lambo Will be interesting to see if imaging without the snapin does make a difference!
Other than that I don’t have other ideas right now. Please give that a try and we’ll see from there.
-
Certainly!
My team an I are working on running without the snapin, as well as updating the Fog client just in case!
I will be in touch.
Have a great day!
Lambo
-
Unfortunately, removing the snapin from the process and upgrading the Fog Client had no change in the outcome. We ran images all day yesterday with the same outcome.
I think I may try deploying a new Fog server and test if it has the same outcome. I don’t have any other ideas on our current server. Do you have any other suggestions?
Thanks!
Lambo
-
@lambo While I do understand the urge to rule out certain things to find out what is causing this I don’t think that setting up a new FOG server will be of any help. This is an issue where the FOG server itself is not involved really. Sure the fog-client polls the AD information from the FOG server DB but after that it’s just the fog-client calling Windows API and talking to your Windows AD server(s). If we would see error codes that tell us credentials are wrong, then we could blame or look into the FOG server but from the evidence we have right now I can’t see the FOG server causing this.
Would you be able/allowed to setup your own AD server for testing?