Fog client installation error - Cannot install CA certificate

[Sebastian Roth]

@jonhwood360 said in Fog client installation error - Cannot install CA certificate:

I wonder if this is a function of it the computer being in Audit mode (ctrl-shift-F3 at OS first boot right after install from media)?

Hmmm, I am not much of a Windows wiz, so can’t say. Would you have an idea @george1421 if that is possible?

[jonhwood360]

@sebastian-roth

So as a test I manually installed the certificate into the certificate store. I confirmed it was in fact installed through the certificate snapin in mmc. When I try to install the client, the certificate disappears from the store once it says it can’t install the certificate.

Pre-install

Post-Install

[george1421]

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

Would you have an idea @george1421 if that is possible?

I’m not sure how much help I can be, because we haven’t used the fog client in over 5 years. When we did use it we would load the service using MDT and then stop and disable the service right away just after it was installed. Then after sysprep and cloning we would restart it in the setupcompleted.cmd. We never touched audit mode because MDT did that part for us. We did use the MSI with command line parameters to install the fog client back then.

So one might wonder what the fog client uses to download the certificate? curl? Could MS have deprecated what the fog client uses to download files?

[Sebastian Roth]

@george1421 said in Fog client installation error - Cannot install CA certificate:

So one might wonder what the fog client uses to download the certificate? curl? Could MS have deprecated what the fog client uses to download files?

The fog-client uses WebClient.DownloadFile() - an officially provided function within the System.Net namespace provided by MS.

A quick search on the web didn’t reveal much about audit mode behaving differently with .NET calls or the cert store. Though I don’t know enough about it…

@jonhwood360 True, the fog-client installer will remove any cert from the store named “FOG Server CA” it finds before it loads the current one from the server to install that. It’s a way of making sure the right CA cert is being installed even if there are left overs from an old install.

Did you manually edit the Apache configuration or left it as generated by the FOG installer?

[Sebastian Roth]

@jonhwood360 Quickly added some debugging output and compiled a fresh MSI for you: https://github.com/FOGProject/fog-client/releases/download/0.12.0/FOGService_debug_CAcert.msi

This is not an official build but it will do a good job finding out what’s going wrong in your case I hope. Try installing with that MSI and then check the FOGService.install.log again. You should see more output in there than you had before. Post the new log output here.

[jonhwood360]

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

Did you manually edit the Apache configuration or left it as generated by the FOG installer?

No I did not.

[jonhwood360]

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

@jonhwood360 Quickly added some debugging output and compiled a fresh MSI for you: https://github.com/FOGProject/fog-client/releases/download/0.12.0/FOGService_debug_CAcert.msi

This is not an official build but it will do a good job finding out what’s going wrong in your case I hope. Try installing with that MSI and then check the FOGService.install.log again. You should see more output in there than you had before. Post the new log output here.

Here you go.

[Sebastian Roth]

@jonhwood360 Didn’t expect it to bail out that early. So it doesn’t even get to where I expected it to fail (SSL/TLS cert validity check).

Could you please try installing the fog-client on a system that is not in audit mode? Just want to make sure this has no effect.

The other thing we might take a look at is a network packet capture. Get the fog-client setup ready to the same point as last time when we looked at the Apache log files. Then run the following commands on your FOG server:

sudo -i
apt install tcpdump
tcpdump -nn -w /tmp/ssl.pcap host 10.40.40.14

Make sure you put in the IP address of the host you are trying to install fog-client on. Now leave the command sit there and finish the fog-client setup. After it failed, stop tcpdump (ctrl-c) and use WinSCP (or another secure copy tool) to copy the binary file /tmp/ssl.pcap over to another computer. Upload to any filesharing service you have access to and post a link here.

[jonhwood360]

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

@jonhwood360 Didn’t expect it to bail out that early. So it doesn’t even get to where I expected it to fail (SSL/TLS cert validity check).

Could you please try installing the fog-client on a system that is not in audit mode? Just want to make sure this has no effect.

The other thing we might take a look at is a network packet capture.

Here is the packet capture:
https://drive.google.com/file/d/1KM4WAsPPF43tVDomDUuR_HOEU_4bZ6oB/view?usp=sharing

[jonhwood360]

@jonhwood360
fog apache config

[Sebastian Roth]

@jonhwood360 The PCAP looks like the host sends a TLS Client Hello using TLS 1.0 and the Apache server rejects it. Have you disabled newer TLS versions via GPO by any chance??

[jonhwood360]

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

@jonhwood360 The PCAP looks like the host sends a TLS Client Hello using TLS 1.0 and the Apache server rejects it. Have you disabled newer TLS versions via GPO by any chance??

No, no GPOs are applied. The apache server is on ubuntu. I can try to force enable newer TLS versions on the workstations. Is ver 1.2 sufficient?

[jonhwood360]

@jonhwood360 said in Fog client installation error - Cannot install CA certificate:

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

@jonhwood360 The PCAP looks like the host sends a TLS Client Hello using TLS 1.0 and the Apache server rejects it. Have you disabled newer TLS versions via GPO by any chance??

No, no GPOs are applied. The apache server is on ubuntu. I can try to force enable newer TLS versions on the workstations. Is ver 1.2 sufficient?

@sebastian-roth,

I reattempted install after hard enabling tls 1.1 and 1.2 in the registry of the machine. No change.

I also took another pcap: https://drive.google.com/file/d/19u1RKug2OwFOHC4S_l0bDT1uK7bbhR0I/view?usp=sharing

[jonhwood360]

@Sebastian-Roth

PCAP from workstation as well - https://drive.google.com/file/d/1y-lML_qrJ18nv3T7HQ3zsW9M9vUD3NOU/view?usp=sharing

[Sebastian Roth]

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

The PCAP looks like the host sends a TLS Client Hello using TLS 1.0 and the Apache server rejects it. Have you disabled newer TLS versions via GPO by any chance??

Sorry, I wrote this without having had the time to test this on my side. I just found the time to do capture a PCAP in my test setup and it seems to use TLS 1.0 as well. Reading more about this on the we I found that it still seems to be the default in .NET framework 4.5.x which we use since fog-client version 0.12.0 (before we still used .NET 4.0, OMG).

Ok that explains why we both see TLS 1.0 in the PCAP but to it’s still a riddle why your Apache server rejects the request while mine doesn’t. Plus we haven’t heard from other users so far. Maybe this Ubuntu moved to some newer versions of Apache (and maybe openssl) not long ago that now reject TLS 1.0 completely.

I will try to force using of TLS 1.1 and newer in our code and upload a new installer for you to test soon.

Thanks a lot for working on this with me! While at first I thought this is not a general issue it seems to actually be and you are the first one to report it.

[Sebastian Roth]

@jonhwood360 Ok, here you go for another try: https://github.com/FOGProject/fog-client/releases/download/0.12.0/FOGService_enable_TLS12.msi

See if it can successfully pin to the FOG server with that and post a picture of the FOGService.install log as well.

Keep in mind, this is not for official deployment for various reasons.

[jonhwood360]

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

@jonhwood360 Ok, here you go for another try: https://github.com/FOGProject/fog-client/releases/download/0.12.0/FOGService_enable_TLS12.msi

See if it can successfully pin to the FOG server with that and post a picture of the FOGService.install log as well.

Keep in mind, this is not for official deployment for various reasons.

We have a winner!

So it seems that newer apache does not like the 1.0 connections.

[Sebastian Roth]

@jonhwood360 said in Fog client installation error - Cannot install CA certificate:

We have a winner!
So it seems that newer apache does not like the 1.0 connections.

Great! Thanks again for testing. I will need to dig through the code and see If there is more adjustment needed (other places in the code needing that fix as well). Allow me a few days and I will release a new 0.12.1 including that fix.

Can you post the latest FOGService.install log as well?

[jonhwood360]

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

@jonhwood360 said in Fog client installation error - Cannot install CA certificate:

We have a winner!
So it seems that newer apache does not like the 1.0 connections.

Great! Thanks again for testing. I will need to dig through the code and see If there is more adjustment needed (other places in the code needing that fix as well). Allow me a few days and I will release a new 0.12.1 including that fix.

Can you post the latest FOGService.install log as well?

As requested:

[Sebastian Roth]

@jonhwood360 Thanks! I didn’t get to it over the weekend but hopefully will this upcoming week.