Posts made by lambo

@sebastian-roth

Hi Sebastian,

No worries at all, i just figured i would ask, as there are multiple domain controllers in our organization and i was hoping there would be a way to point fog to utilize the local DC. No worries though at all!

After testing with the Server 2019 DC in production, it seems that some of the issues presented have been resolved in some regard.

Here is what we found:

If a computer AD account already exists, it seems we can reimage with no issues now. we tested around 15 images and they all worked successfully. We can also reimage a computer where the AD account has been ‘reset’ instead of deleted. These tests were successful as well.

However, if we delete the computer AD account or image a new PC and have the option to add the computer to the domain, the issue we were previously experiencing exposes itself. The computer will allow AD accounts to log in, but trying to deploy software shows a trust relationship issue as well as having the computer nowhere to be found in Active Directory.

What would you like to see done here? I am about out of ideas on our end unfortunately, other than standing up a test domain and seeing if the issue persists there and if so, trying to narrow down where the problem is. I think we should be able to set this up and we can continue deeper testing there.

Thanks!

Lambo

@sebastian-roth

Hi Sebastian,

I hope all is well with you!

Sorry for the delay.

Ok so we were able to produce the AD issue with the AD team.

Unfortunately, we didn’t see much happening in terms of event logs.

There were three separate failures that occurred, but not necessarily related to joining the domain in my experience. I could certainly be mistake though.

1. Audit Failure - Kerberos Authentication Service - 4786

2. Audit Failure - Directory Service Access - 4662

3. Audit Failure - Credential Validation - 4776

None of these failures utilized our image account for Fog / Hostname Changer, so I don’t see them as being a smoking gun unfortunately.

The AD Team was planning a migration for our AD server to Windows Server 2019, so they will be completing this ahead of schedule for us. This migration will take place this week, so we are hopeful that it could assist with resolving the issues that we are seeing.

If that does not work, the AD team will allow us to utilize a test AD server for troubleshooting this issue. We should then be able to fix up and test any settings we think are important.

Do you also know if it is possible to ‘lock’ the domain controller to a specific domain controller in Fog? This would be helpful so we could lock Fog in to utilizing our local domain controller instead of possibly utilizing another in the network.

Again, sorry for the delay, and thank you so much. I really appreciate your assistance!

Thanks!

Lambo

@sebastian-roth

I just wanted to update you Sebastian.

This Wednesday / Thursday I will be working with the AD team to view logs during the imaging processes. They are most likely going to request that I have access to set up a temporary AD Server / get access to their Test AD servers.

I am hoping to have an update for you after testing this week.

Please let me know if you need anything else.

Do you also have a link to the Fog recommended image process so i can ensure it is not an issue with our image as well? Our Windows version is Win 10 Enterprise 20H2.

Thanks!

Lambo

@sebastian-roth

That makes sense! I was thinking along the lines of a corruption or similar with this specific issue.

I can check with our higher ups to see if we could create a local AD server for testing purposes to see if we would be able to work with this server on narrowing down the issues that are presenting themselves. Let me get back to you on this point and let you know if this is possible.

Thanks!

Lambo

@sebastian-roth

Unfortunately, removing the snapin from the process and upgrading the Fog Client had no change in the outcome. We ran images all day yesterday with the same outcome.

I think I may try deploying a new Fog server and test if it has the same outcome. I don’t have any other ideas on our current server. Do you have any other suggestions?

Thanks!

Lambo

@sebastian-roth

Certainly!

My team an I are working on running without the snapin, as well as updating the Fog client just in case!

I will be in touch.

Have a great day!

Lambo

@sebastian-roth

Ok, that is understandable. Is there anything that can verify the installation?

I will work on the Fog client and seeing if upgrading changes anything, as well as imaging without the snapin enabled.

Do you have any other suggestions?

Thanks!

Lambo

@sebastian-roth

Sorry for the delay, we have a major project that I am currently involved with in my organization.

I agree! But there must be a cause whether its on our end or something that may have gotten corrupted on our server. This whole issue is quite strange and is very perplexing. I can’t seem to find any rhyme or reason.

I will have a look at these error codes and see if I can’t find anything that would be causing this issue.

The only thing that we have is a snapin for our software deployment software that is set to run after the PC is imaged and on the domain. I can certainly try to run some images without this snapin enabled and see if that makes a difference?

Other than that, i am not seeing anything that could cause issues like we are seeing. I’m not seeing too much in the event logs and I don’t have access to the AD server, but i had the team look at it and they said they haven’t seen anything unusual.

Let me take a look at the Fog client as well, even though you might not think it could be the issue, we can try to ensure that it isn’t causing some funky issue.

Do you suggest maybe downgrading the server version to 1.5.8 as well just to test and ensure we didn’t encounter some weird issue when upgrading, perhaps something got corrupted on our end or similar?

Thanks, sorry again for the delay!

Lambo

@sebastian-roth

Here is a link you can try, Let me know if you are able to access the drive.

https://drive.google.com/drive/folders/1aOwYRK71pVZL68uIrzPzT1ylOmquVzeP?usp=sharing

Fog logs are listed, Testing Notes.txt are the overall notes showing what tasks have been taken for each fog log.

Let me know if you need anything else!

Thanks!

Lambo

@sebastian-roth

Ok i Have the data available!

Would you like me to put the summary and all 10 Fog logs right here or is there a different way you prefer I upload these files?

Thanks!

Lambo

@Sebastian-Roth

Hi Sebastian,

Just wanted to update you that I am combing through the data that I have on the imaging sequences that we spoke about last.

I will be able to upload the process and data to you shortly!

Thanks!

Lambo

@sebastian-roth

Thank you for that tip! I will ensure this gets done as well.

It may take me some time to get this data to you with our hybrid schedule, but I will be sure to upload it!

I am very interested to see what we get as well. I will also comb through these logs to ensure that there isn’t anything apparent in that area as well.

Thanks!

Lambo

@sebastian-roth

Hi Sebastian,

Sorry for the delay, we have been quite busy at work.

You are exactly correct, the known good image has the same issue as well. No MS updates on it.

I will certainly test this out! I think this is a great idea and hopefully it produces something tangible we can utilize.

I will report back once finished!

Thanks!

Lambo

@sebastian-roth

Thank you so much for your reply, I have seen your name dozens of times while researching various issues. It is a pleasure to speak with you.

Can you be more specific on when this started? Did it work without issues before? Just wondering if it could be Windows or .NET updates or something else that brought this issue up.

This issue was discovered on 1/28/2021 in our organization. We had new PCs to deploy and some technicians discovered the issue, but I am unsure how long it would have existed before we discovered the issue. AD Join / Hostname Changer did work perfectly fine before this issue was discovered.

It is quite possible that this issue is because of a newer .NET issue or similar, it is hard to say with all the changes MS is implementing! We did have a new image created, Version 20H2, but our old image is experiencing issues as well, Version 2004. I can troubleshoot whatever you need though!

None of those things would play a role in this case. Not the kernel used and there have not been changes in how FOG handles hostname changes between 1.5.9 and 1.5.9.60. As well, the fog-client being involved a lot has not been updated.

Did you update from an earlier version to 1.5.9 when this started to happen?

I didn’t think these would play a role in Hostname Changer, but I figured I would give it a try just in the off chance that there was a difference. We upgraded from v1.5.8 to v1.5.9. From my memory, we didn’t have any issues after the upgrade. But see if anyone else noticed anything.

If the fog-client version used has not changed I can only see other components (Windows build version, Windows Updates, .NET Updates, AD server Updates, GPO, …) causing this.

Don’t get me wrong. I am not saying this is non of our business. What I am asking for is a detailed description of components that changed since it worked last. Otherwise this won’t get us anywhere I suppose.

I can certainly see how that could affect the system as well. Give me some time to try and curate a list of these changes, although the issue has appeared on our known good image as well. But I can still compile a list.

Which Windows 10 build version do you use?

Previously we were utilizing 2004, then we moved to 20H2. We always keep one known good version in the event there are issue with the new image.

Which AD server version?

AD Server version is Server 2012 R2

All current updates installed on the systems?

At the time of creation all updates were installed on the image, but i can update a PC after imaging or update the image if you’d like.

Just to clarify what this option really does is that it edits the Windows registry files after deploying the image to disk (before the very first reboot!) to change the computer name. This has worked with Windows 7 in the past and I have tested this with Windows 10 not long ago. Though I did not test this in a setup with AD integration. So I can’t say if this would cause any side effects.

That’s an interesting point! From my point of view there is no need to have CHANGE HOSTNAME EARLY enabled if you use the fog-client doing the renaming for you. As you say it shows less issues, just leave that switched off.

Thank you so much for the information on this setting! I wasn’t sure exactly how it functioned. Unfortunately for me, today it made me a liar, in my testing it isn’t wanting to add the PC to AD even with this setting unchecked. We are just experiencing very very strange issues. It also acts like the PC is on the domain, I can log in with my network account, but the PC is nowhere on the domain and will eventually encounter a Trust Relationship issue.

Let me know if there is anything at all you would be interested in seeing. I can recreate the images, grab logs, test whatever you may need.

One thought I had, is it possible to downgrade from version 1.5.9 to 1.5.8 for testing purposes?

Thanks!

-Lambo

Hello,

I wanted to post this in general as I am unsure of the correct area to post issues with Hostname Changer.

Currently, we just started experiencing issues with Hostname Changer over the past couple of weeks. There has been no maintenance done on the Fog Server, No image changes, and no changes on the back end previous to the issues we started to experience.

We are running FOG Version 1.5.9 with Debian 10. We have also tried a couple of the trunk versions as well for troubleshooting this issue, including 1.5.9.60. We also tested with the latest Kernel build. None of these seem to have resolved the issues we are experiencing, nor did they have any change in the issue we are experiencing.

The issue that we have started to see somewhat broad, but I will explain as best I can.

1. On new images, if the PC doesn’t exist Hostname Changer will no longer create an AD Object for the PC.

2. Sometimes imaging a computer using the name of a pre-existing AD object will cause additional trust relationship issues, but FOG will act like it’s on the domain.

   Ex: We had a computer named TEST that we are replacing with a new device. The device images and successfully renames to TEST. Initially we are able to access the computer with a domain account, but after a few moments it will receive a trust relationship error. FOG.log will show the device is joined to the domain with code 2691. Using Control Panel, we can rename the computer to TEST-01 and put it into a workgroup. FOG will then rename the device back to TEST and add it to the domain, but we’ll receive the same error. We can rename the computer in FOG to TEST-01 and it’ll add it in AD as a new object. We can then delete the old TEST object and rename it again using FOG. That will typically allow the computer to work correctly under the name TEST.

3. Renaming a computer does not create a new AD object. However, it is still able to access the domain (no sign issues or trust relationship errors). Adding a new computer with the previous name will cause trust relationship issues on the new computer.

4. Sometimes renaming a PC WILL create a new AD object, which is different from number 3 above.

These are just some of the scenarios that we are experiencing, I can list them all but there are a vast number of scenarios that occur.

One thing I did find on an old forum post, is the following setting:

Fog Configuration > General Settings > CHANGE HOSTNAME EARLY

Turning this off allowed new AD objects to be created successfully on new images with no preexisting AD entry. But a plethora of other issues still exist with Hostname Changer that didn’t exist before.

The strange thing is, this setting has been on and in place for years without issue.

We have also verified the domain account being utilized is active and has all permissions and attributes necessary to successfully add PCs to the domain. We tested this manually with a few PCs with 100% success.

Any thoughts or suggestions on where to look or troubleshoot would be appreciated!

The Fog Log shows the PC as on the domain in most of these cases with no errors.

Thanks!

-Lambo