Certificate issues since moving FOG from Ubuntu to Fedora.

Hanz

I have Fedora 22 running SVN 5315 now…I’ve had previous verisons of SVN running on Ubuntu, until I just got fed up with the “workarounds” Now I have most clients running 9.6 client, but I’m concerned with “invalid security token” and “certificate not from FOG CA” breaking things moving forward. I’m not sure what to do as I’ve uninstalled/re-installed fogservice (some wouldn’t even let me reinstall), reset encryption data, re-pinned via @Jbob fix, there has to be an easier way to just remove fogservice (gpo won’t actually uninstall, even after several reboots) and start over somehow. Can anyone please advise…I have Teamviewer and am willing to let a developer look sometime during my work hours. Thanks

Wayne Workman

@Developers

I took a look at this.

msiexec /x '{ 9.4 product code here }' /q would not get the old client off.

It did not show up in programs and features either but did show up in the report from wmic product get > products.txt

The new client would not install manually.

After toying around for a little while, I re-installed the 9.4 client using the MSI that @Hanz had and it did install OK, and then I removed it via programs and features and it did remove ok and no longer showed up in the wmic product get > products.txt output.

After that, we tried installing the 9.6 client and rebooted but… the remote computer we were working on never came back up… so… I don’t know how it ended up…

Hanz

@Wayne-Workman Unfortunately it looks like my old server was “re-purposed” for a lab, so no chance to retrieve old ssl keys. Is there any way to remove the old keys from machines and install the newly generated keys, from new Fedora based server ?

Tom Elliott

You can use one of the utilities Jbob has created (you may need to contact him).

He can also help create a snapin to make sure all the hosts that have the new client will get the clients to repin the certificate according to the new server ca certs.

Hanz

@Tom-Elliott FYI SVN 5315 doesn’t have the location setting under groups. Thanks, I have the repin .exe, so I will attempt to get clients repinned…Do I need to reset encryption data first ?

Hanz

@Tom-Elliott Doesn’t look like snapins are going to deploy. This is even after resetting encryption data, unless it takes until next check-in to reset encryption data.

------------------------------------------------------------------------------
---------------------------------SnapinClient---------------------------------
------------------------------------------------------------------------------
 11/13/2015 9:53 AM Client-Info Version: 0.9.6
 11/13/2015 9:53 AM SnapinClient Running...
 11/13/2015 9:53 AM Middleware::Communication URL: http://10.72.3.50/fog/service/servicemodule-active.php?moduleid=snapinclient&mac=B4:99:BA:E9:B8:B8|0A:00:27:00:00:00|CC:52:AF:87:F3:DA||00:00:00:00:00:00:00:E0|00:00:00:00:00:00:00:E0&newService=1
 11/13/2015 9:53 AM Middleware::Communication Response: Success
 11/13/2015 9:53 AM Middleware::Communication URL: http://10.72.3.50/fog/service/snapins.checkin.php?mac=B4:99:BA:E9:B8:B8|0A:00:27:00:00:00|CC:52:AF:87:F3:DA||00:00:00:00:00:00:00:E0|00:00:00:00:00:00:00:E0&newService=1
 11/13/2015 9:53 AM Middleware::Communication Response: Invalid host certificate
 11/13/2015 9:53 AM Middleware::Communication URL: http://10.72.3.50/fog/management/other/ssl/srvpublic.crt
 11/13/2015 9:53 AM Data::RSA FOG Server CA cert found
 11/13/2015 9:53 AM Middleware::Authentication Cert OK
 11/13/2015 9:53 AM Middleware::Communication POST URL: http://10.72.3.50/fog/management/index.php?sub=authorize
 11/13/2015 9:53 AM Middleware::Communication Response: Invalid security token
------------------------------------------------------------------------------```

Tom Elliott

@Hanz I’d recommend resetting the encryption data, yes.

Tom Elliott

@Hanz Location for groups is now fixed.

Hanz

@Tom-Elliott Thanks, @Jbob instructed me to

“Just wipe all the security tokens
That will fix it”

Wayne Workman

@Hanz lol. @Jbob can you please elaborate?

Hanz

Per Jbob I ran the following cmd on fog database to clear security tokens.

UPDATE hosts SET hostPubKey="", hostSecToken="", hostSecTime="0000-00-00 00:00:00";

I also reset encryption data from Web UI on all hosts (which I assume does the same things from different angles)

This seemed to work, as of now my VM host is working correctly (although it has not been up and running as long as my work comp)
BUT
My work computer which has been left on, has gone from “Authentication Authenticated” back to “Invalid host certificate” and “Invalid Security Token”

I restarted the service on the VM in order to force the update to 9.7 from 9.6 and it went through the process without a hitch, through reboots and all. Im running it all day today to see if I can find the point it fails (if it does).

Wayne Workman

@Hanz said:

Per Jbob I ran the following cmd on fog database to clear security tokens.
UPDATE hosts SET hostPubKey="", hostSecToken="", hostSecTime="0000-00-00 00:00:00";
I also reset encryption data from Web UI on all hosts (which I assume does the same things from different angles)

This seemed to work, as of now my VM host is working correctly (although it has not been up and running as long as my work comp)
BUT
My work computer which has been left on, has gone from “Authentication Authenticated” back to “Invalid host certificate” and “Invalid Security Token”

I restarted the service on the VM in order to force the update to 9.7 from 9.6 and it went through the process without a hitch, through reboots and all. Im running it all day today to see if I can find the point it fails (if it does).

Added to WiKi: https://wiki.fogproject.org/wiki/index.php/FOG_Client

Hanz

0_1447629009513_fog.log

This is a copy of my log on the machine I mentioned that keeps losing security token somehow…at the 1:35 pm mark it shows invalid host certificate, but an Authentication Authenticated statement…

On the next checkin ~ 2:36 it goes to invalid host certificate, invalid security token.

I reset encryption data (again) and restarted service on the local computer @ the 6:00 mark and the final shows it going back to Authentication Authenticated upon restart of the service. (sorry for the uploaded log, but it wouldn’t let me post just the copied code this time.)

This is next checkin

------------------------------------------------------------------------------
----------------------------------TaskReboot----------------------------------
------------------------------------------------------------------------------
 11/15/2015 7:03 PM Client-Info Version: 0.9.7
 11/15/2015 7:03 PM TaskReboot Running...
 11/15/2015 7:03 PM Middleware::Communication URL: http://10.72.3.50/fog/service/servicemodule-active.php?moduleid=taskreboot&mac=B4:99:BA:E9:B8:B8|0A:00:27:00:00:00|CC:52:AF:87:F3:DA||00:00:00:00:00:00:00:E0|00:00:00:00:00:00:00:E0&newService=1
 11/15/2015 7:03 PM Middleware::Communication Response: Success
 11/15/2015 7:03 PM Middleware::Communication URL: http://10.72.3.50/fog/service/jobs.php?mac=B4:99:BA:E9:B8:B8|0A:00:27:00:00:00|CC:52:AF:87:F3:DA||00:00:00:00:00:00:00:E0|00:00:00:00:00:00:00:E0&newService=1
 11/15/2015 7:03 PM Middleware::Communication Response: Invalid host certificate
 11/15/2015 7:03 PM Middleware::Communication URL: http://10.72.3.50/fog/management/other/ssl/srvpublic.crt
 11/15/2015 7:03 PM Data::RSA FOG Server CA cert found
 11/15/2015 7:03 PM Middleware::Authentication Cert OK
 11/15/2015 7:03 PM Middleware::Communication POST URL: http://10.72.3.50/fog/management/index.php?sub=authorize
 11/15/2015 7:03 PM Middleware::Communication Response: Success
 11/15/2015 7:03 PM Middleware::Authentication Authenticated
 11/15/2015 7:03 PM Middleware::Communication URL: http://10.72.3.50/fog/service/jobs.php?mac=B4:99:BA:E9:B8:B8|0A:00:27:00:00:00|CC:52:AF:87:F3:DA||00:00:00:00:00:00:00:E0|00:00:00:00:00:00:00:E0&newService=1&newService=1
 11/15/2015 7:03 PM Middleware::Communication Response: No jobs
------------------------------------------------------------------------------```

Hanz

@Wayne-Workman After clearing all security tokens for hosts, my database shows all hosts with no tokens…When are these tokens supposed to recreate themselves, as it looks like none are being recreated which may be why my clients keep saying invalid host certificate. I don’t know who creates them or when for that matter, but it doesn’t seem to be happening.

Tom Elliott

Fog automatically creates the token during the authentication sequence. Ihc is a signifier to the client that a new aes key needs to be generated. If the aes key and security token are blank the server creates a security token for the client and the client creates its own aes key. The server stores the aes key with the host for a specified period of time (30 minutes for now) and resets the key to null if the expired time occurs. When the key expires the client will receive the ihc (invalid host certificate) and it knows it needs to generate a new aes key. During every authentication sequence, as spawned when ihc is met, (after initial connect) the client sends what it knows is the current security token. As long as this matches what the server knows is true, a new security token is generated and sent to the client and the server stores the newly generated aes key for the host.

Hanz

@Tom-Elliott As of now I have no hosts with any security tokens or pubkeys associated, this is after resetting them Friday. Some have been running all weekend. All I have under /opt/fog/snapins/ssl is a file named fog.csr. The client doesn’t seem to be creating new aes keys, seeing as how they’re all “expired”/blank on server. Am I getting that right ?

Tom Elliott

@Hanz if you run:
ls -lhart /opt/fog/snapins/ssl do you see a .srvprivate.key?

Hanz

@Tom-Elliott yes

Hanz

@Hanz sorry didn’t realize it was a hidden file when Wayne Workman showed me where they were.

Hanz

@Tom-Elliott below is the output from said command. Curious about the permissions for the … as it usually is owned by root I thought.

[bcs@fog-server ~]$ ls -lhart /opt/fog/snapins/ssl
total 16K
drwsrwsrwx 2 fog apache 4.0K Oct 16 09:25 .
drwsrwsr-x 4 fog apache 4.0K Nov 15 19:34 ..
-rwxrwxrwx 1 fog apache 3.2K Nov 15 23:35 .srvprivate.key
-rwxrwxrwx 1 fog apache 1.6K Nov 15 23:35 fog.csr

Currently running 5368

Certificate issues since moving FOG from Ubuntu to Fedora.

258

12.0k

17.3k

155.2k