Trouble installing SmartClient on MacOS Sierra

Daniel Miller

Good News:
The installer finished up without error.
After rebooting the machine and logging back into the admin user, the tray icon appeared, the service process appears to be running, and /opt/fog-server/fog.log appears to be initially happy:

 6/28/2019 11:29 AM Middleware::Communication URL: http://fogserver/fog/management/index.php?sub=requestClientInfo
&configure&newService&json
 6/28/2019 11:29 AM Middleware::Response Success
 6/28/2019 11:29 AM Middleware::Communication URL: http://fogserver/fog/management/index.php?sub=requestClientInfo
&mac=00:25:00:F0:83:DF|00:25:00:F0:6F:45|00:25:4B:FF:FE:FB:69:24&newService&json
 6/28/2019 11:29 AM Middleware::Response Invalid host
 6/28/2019 11:29 AM Middleware::Communication URL: http://fogserver/fog/service/getversion.php?clientver&newServic
e&json
 6/28/2019 11:29 AM Middleware::Communication URL: http://fogserver/fog/service/getversion.php?newService&json

 6/28/2019 11:29 AM Service Creating user agent cache
 6/28/2019 11:29 AM Middleware::Response Module is disabled on the host
 6/28/2019 11:29 AM Middleware::Response Module is disabled on the host
 6/28/2019 11:29 AM Middleware::Response Module is disabled globally on the FOG server

Bad News:
Certificate store appears to be ~/.config/.mono/certs/Trust with respect to the installing user account and the service doesn’t appear to be reliably pulling from that store after further reboots. From /opt/fog-service/fog.log:

------------------------------------------------------------------------------
--------------------------------Authentication--------------------------------
------------------------------------------------------------------------------
 6/28/2019 11:44 AM Client-Info Version: 0.11.16
 6/28/2019 11:44 AM Client-Info OS:      Mac
 6/28/2019 11:44 AM Middleware::Authentication Waiting for authentication timeout to pass
 6/28/2019 11:46 AM Middleware::Communication Download: http://fogserver/fog/management/other/ssl/srvpublic.crt
 6/28/2019 11:46 AM Data::RSA ERROR: Unable to retrieve FOG Server CA
 6/28/2019 11:46 AM Data::RSA ERROR: FOG Server CA NOT found in keystore
 6/28/2019 11:46 AM Middleware::Authentication ERROR: Could not authenticate
 6/28/2019 11:46 AM Middleware::Authentication ERROR: Value cannot be null.
Parameter name: authority


 6/28/2019 11:46 AM Middleware::Communication URL: http://fogserver/fog/management/index.php?sub=requestClientInfo&configure&newService&json
 6/28/2019 11:46 AM Middleware::Response Success
 6/28/2019 11:46 AM Middleware::Communication URL: http://fogserver/fog/management/index.php?sub=requestClientInfo&mac=00:25:00:F0:83:DF|00:25:00:F0:6F:45|00:25:4B:FF:FE:FB:69:24&newService&json
 6/28/2019 11:46 AM Middleware::Authentication Waiting for authentication timeout to pass
 6/28/2019 11:48 AM Middleware::Communication Download: http://fogserver/fog/management/other/ssl/srvpublic.crt
 6/28/2019 11:48 AM Data::RSA ERROR: Unable to retrieve FOG Server CA
 6/28/2019 11:48 AM Data::RSA ERROR: FOG Server CA NOT found in keystore
 6/28/2019 11:48 AM Middleware::Authentication ERROR: Could not authenticate
 6/28/2019 11:48 AM Middleware::Authentication ERROR: Value cannot be null.
Parameter name: authority
 6/28/2019 11:48 AM Middleware::Response Success
 6/28/2019 11:48 AM Middleware::Communication URL: http://fogserver/fog/service/getversion.php?clientver&newService&json
 6/28/2019 11:48 AM Middleware::Communication URL: http://fogserver/fog/service/getversion.php?newService&json

 6/28/2019 11:48 AM Service Creating user agent cache
 6/28/2019 11:48 AM Middleware::Response ERROR: Unable to get subsection
 6/28/2019 11:48 AM Middleware::Response ERROR: Object reference not set to an instance of an object
 6/28/2019 11:48 AM Middleware::Response ERROR: Unable to get subsection
 6/28/2019 11:48 AM Middleware::Response ERROR: Object reference not set to an instance of an object
 6/28/2019 11:48 AM Middleware::Response ERROR: Unable to get subsection
 6/28/2019 11:48 AM Middleware::Response ERROR: Object reference not set to an instance of an object
 6/28/2019 11:48 AM Service Initializing modules

Being the service appears to start as root, I’m not entirely certain how it was initially able to access the cert store (I blame unicorns). Unfortunately, the constructor for X509Store appears to be doing exactly what it was told without any pleasant side effects.

Additionally, even when the service was checking in, the fog server didn’t appear to be acknowledging the communications; the http requests were showing up in other_vhosts_access.log on the fog server, but no pending registrations appeared in the web interface and, when I manually added the host and scheduled a hardware inventory, no task reboot information appeared to be passed to the client. This may be attributable to the lack of CA (or the aforementioned unicorns).

I did try swapping out the Zazzles.dll with the one posted in Client not authenticating, but it is giving the same results.

Sebastian Roth

@Daniel-Miller Well at least looks like we are making a bit of progress here. Not perfect but still seems like we got some space to try things out.

First I might ask you to try and run the installer as root and see if you can make it work that way.

 6/28/2019 11:46 AM Data::RSA ERROR: Unable to retrieve FOG Server CA
 6/28/2019 11:46 AM Data::RSA ERROR: FOG Server CA NOT found in keystore

This is most probably due to it not finding the cert store in the right place. No chance to make it work using the Zazzles.dll I patched for Linux.

Daniel Miller

@Sebastian-Roth Nice call.
Installation, both under the auspices of a sudo su -l in terminal and as root through ARD, appear to be behaving after a manual kick to launchctl. The newest iMacs don’t seem to be triggering a pending host notification, but they are logging the communication on the client and in the apache logs. I suspect there might be too many mac addresses, but I am thinking that is a Monday problem.

Sebastian Roth

Cross linking a new post on the same issue: https://forums.fogproject.org/topic/13611/unable-to-complete-smartinstaller-exe-on-macos-mojave-10-14-6

Probably something we should try to fix but it’s a tough one, I don’t have Mac OS X to test and very little time. @Daniel-Miller Are you willing and keen to setup a fog-client development environment and help work on this?

Daniel Miller

@Sebastian-Roth I have a couple units I can probably play with and might be able to find a little time to cobble something together.

Sebastian Roth

@Daniel-Miller Any news on this topic?

Daniel Miller

@Sebastian-Roth
Start of semester … time is an illusion
It is just a matter of relaunching the installer if on a mac and not running as ‘root’ in addition to the change you made. That was just substituting StoreLocation.CurrentUser for StoreLocation.LocalMachine in GenericSetup.cs, yes? I have some logic framed in for the relaunch that puts it in the right user context, but something about how the installer is built isn’t getting the auxiliary files (.sh, .dll, etc.) packed with the executable, so while the relaunched environment looks right, I can’t definitively say it works. Can drop the diff here unless you have a better means of getting things from point A to B.

Sebastian Roth

@Daniel-Miller I have read you message at least four times now but I still don’t fully understand.

That was just substituting StoreLocation.CurrentUser for StoreLocation.LocalMachine in GenericSetup.cs, yes?

Yes I think so. Just checked the diff I had lying around:

diff --git a/SetupHelper/GenericSetup.cs b/SetupHelper/GenericSetup.cs
index a9b7a72..d4e2f41 100644
--- a/SetupHelper/GenericSetup.cs
+++ b/SetupHelper/GenericSetup.cs
@@ -110,7 +110,7 @@ namespace FOG

             try
             {
-                var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
+                var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
                 store.Open(OpenFlags.ReadWrite);
                 store.Remove(cert);
                 store.Close();
@@ -129,7 +129,7 @@ namespace FOG
             try
             {
                 var cert = new X509Certificate2(location);
-                var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
+                var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
                 store.Open(OpenFlags.ReadWrite);
                 var cers = store.Certificates.Find(X509FindType.FindBySubjectName, "FOG Project", true);

@@ -168,7 +168,7 @@ namespace FOG
             try
             {
                 X509Certificate2 CAroot = null;
-                var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
+                var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
                 store.Open(OpenFlags.ReadOnly);
                 var cers = store.Certificates.Find(X509FindType.FindBySubjectName, "FOG Project", true);

@@ -191,7 +191,7 @@ namespace FOG

             try
             {
-                var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
+                var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
                 store.Open(OpenFlags.ReadWrite);
                 store.Remove(cert);
                 store.Close();

but something about how the installer is built isn’t getting the auxiliary files (.sh, .dll, etc.) packed with the executable

Can you be more specific on what’s going wrong here or why you think the files are not packed?

Which version are you trying to build from? Latest master? Since we talked about this last in June (!) I have pushed a commit to update dependencies. This is causing a bit of trouble and I have not found the time to properly get this right again. Please use commit b2995511 for now! Sorry for that.

You might also want to change the build script to get more output:

diff --git a/build.ps1 b/build.ps1
index 0773e88..f15bfc0 100644
--- a/build.ps1
+++ b/build.ps1
@@ -118,7 +118,7 @@ Copy-Item "$PSScriptRoot\bin\FOGService.msi" $InstallerMSI
 Copy-Item "$PSScriptRoot\bin\FOGService.msi" "$PSScriptRoot\out\FOGService.msi"

 Write-Host "Building Smart Installer"
-Invoke-Expression ($msbuild + $installerConfig) | out-null
+Invoke-Expression ($msbuild + $installerConfig) #| out-null


 Write-Host "ILMerging Smart Installer"

Daniel Miller

@Sebastian-Roth
That diff matches the one I had for GenericSetup.cs.

Lots of little things, like Visual Studio complaining about the name of OSX-FOG-TRAY.ZIP (renamed it to make it happy, but don’t know if I found all the references), SmartInstaller.exe only being 4.5 MB, and initial error messages about not being able to find Zazzles.dll when running the executable. Resynches off the current master and that commit seems to do the same thing, so I’m almost certain it’s something wrong with how my environment is set up and very likely due to MS not distributing ilmerge any more. Does this GitHub for ILMerge look like the same beast?

And this is all assuming I am actually synching to the points I am intending to … I’m a little new to using source control.

Sebastian Roth

@Daniel-Miller said:

That diff matches the one I had for GenericSetup.cs.

Good.

Lots of little things, like Visual Studio complaining about the name of OSX-FOG-TRAY.ZIP (renamed it to make it happy, but don’t know if I found all the references)

Right, I keep forgetting about this one. Should have fixed that a long time ago. I usually rename it after cloning the fresh source and soon forget about it. It’s good others look into this as well.

SmartInstaller.exe only being 4.5 MB, and initial error messages about not being able to find Zazzles.dll when running the executable. Resynches off the current master and that commit seems to do the same thing, so I’m almost certain it’s something wrong with how my environment is set up and very likely due to MS not distributing ilmerge any more. Does this GitHub for ILMerge look like the same beast?

Yes, sounds very much like it. Something I have not noticed yet as we have our build environment set up since a long time. Good three years ago when I updated the build instrcutions in README.md the download from Microsoft still was available. The one you found in github looks very promising but seems like there are no official MSI releases that you can simply install as it used to. I will look into that and get back to you in a few hours.

And this is all assuming I am actually synching to the points I am intending to … I’m a little new to using source control.

What you usually do on the command line - and I assume you have the same options in a GUI GIT tool:

Clone a copy of the repo - you can skip that as you’ve already done that: git clone https://github.com/FOGproject/fog-client
Now the repo is on master branch latest version and you want to checkout a different version: cd fog-client; git ccheckout b2995511
Open the project in Visual Studio - or reload if you had it open already!

Sebastian Roth

@Daniel-Miller Have not found the time to work on this but doing a bit more search engine foo I found this: https://github.com/mono/nuget/blob/master/Tools/ILMerge/ILMerge.msi

You might use that older version of ILMerge till we get the other one ready.

Daniel Miller

@Sebastian-Roth Seems to be a lot happier on the build side, but the exe is still a bit anemic, only 6.2MB as opposed to the 8.9MB that is on my servers. It does run and properly relaunches, so the requisite dlls are there, but fails during the Installing Files stage with errors in the log

 9/11/2019 4:59 PM Installer ERROR: Method: <Install>b__0
 9/11/2019 4:59 PM Installer ERROR: Central Directory corrupt.

I would guess the service scripts, tray executables, etc. still aren’t getting bundled in.

Looking at the build script, I don’t see where they get attached, or are those supposed to be configured as resources in the properties for UniversalInstaller?

Sebastian Roth

@Daniel-Miller Yeah, still sounds like there is something missing. Try removing the | out-null at the end of the lines to get more output from the commands in build.ps1. As well you might want to try this if you have a Windows PC at hand: Copy the SmartInstaller.exe over to that PC and use 7-Zip to extract the EXE file. Within the extracted files you find one that is several MB in size called .text. Again, rightclick that file and use 7-Zip to extract it. It will do some extraction but also print an error message. But again you should have files extracted. How many files does it extract from your binary on the second run?

9/11/2019 4:59 PM Installer ERROR: Method: <Install>b__0
9/11/2019 4:59 PM Installer ERROR: Central Directory corrupt.

I haven’t seen this error before. I’ve dug through the code but can’t seem to figure out why it says b__0!? When extracting manually as described above you get filenames like Zazzles and so on.

Daniel Miller

@Sebastian-Roth Never seen that error before, you say? Now don’t I feel special.
Extracting .text gives me 50 files and they seem to reasonably match (by filename and checksum) the 48 extracted from production version. Extra files are a Modulespt and ShutdownGUIpt.

Google-foo indicates the “Central Directory corrupt” error results from some interaction with a zip archive and the b__0 might be a reference to some helper class. Only place I’m seeing in the install process that might bump into that would be the extraction of osxfogtray.zip. I still don’t see where that zip gets bundled in either the working build or the production build, but just for ho-hos I decided to take a peek at what that file looks like and it appears to be 0 bytes in size. As that probably isn’t right for a zip archive, I checked the repo and oddly enough the file ./UniversalInstaller/Scripts/OSX-FOG-TRAY.zip is 0 bytes in size and it looks like it has been that way since the initial commit.

Sebastian Roth

@Daniel-Miller Ohhh gosh!! I see our build process is pretty well messed up. Sorry for all the issues you run into and wast all your time on.

Please download the OSX Tray ZIP here for now: https://github.com/FOGProject/fog-client/releases/download/0.11.16/OSXFOGTray.zip

Thank you very much for sticking around even though things are not going very well. We just don’t have enough people to help out and work on all the different FOG components and so things get messy - too bad.

Daniel Miller

@Sebastian-Roth it’s almost like an open source project depends on free labor or something …

All right, still don’t see how that gets brought it, but it is building, a healthy 8.75 MB, and the installation process is now happy with the install files segment. But now we’re back to the Cert Pinning issue, however (names changed to protect the guilty).

 9/12/2019 4:28 PM Data::RSA ERROR: Unable to retrieve FOG Server CA
 9/12/2019 4:28 PM Data::RSA ERROR: FOG Server CA NOT found in keystore
 9/12/2019 4:28 PM Middleware::Communication Download: http://fogserver/fog/management/other/ca.cert.der
 9/12/2019 4:28 PM Data::RSA Injecting root CA: 
 9/12/2019 4:28 PM Data::RSA ERROR: Unable to inject CA
 9/12/2019 4:28 PM Data::RSA ERROR: Access to the path '/usr/share/.mono/certs/Trust' is denied.

Which brings us back to GenericSetup and the cert store. Currently the diff between my current and commit b299551 is:

@@ -108,11 +108,11 @@ namespace FOG
             var cert = RSA.ServerCertificate();
             if (cert == null) return false;
 
             try
             {
-                var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
+                var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
                 store.Open(OpenFlags.ReadWrite);
                 store.Remove(cert);
                 store.Close();
                 return true;
             }
@@ -127,11 +127,11 @@ namespace FOG
         public static bool InstallFOGCert(string location)
         {
             try
             {
                 var cert = new X509Certificate2(location);
-                var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
+                var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
                 store.Open(OpenFlags.ReadWrite);
                 var cers = store.Certificates.Find(X509FindType.FindBySubjectName, "FOG Project", true);
 
                 var validKeyPresent = false;
                 if (cers.Count > 0)
@@ -166,11 +166,11 @@ namespace FOG
         {
             var cert = new X509Certificate2();
             try
             {
                 X509Certificate2 CAroot = null;
-                var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
+                var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
                 store.Open(OpenFlags.ReadOnly);
                 var cers = store.Certificates.Find(X509FindType.FindBySubjectName, "FOG Project", true);
 
                 if (cers.Count > 0)
                 {
@@ -189,11 +189,11 @@ namespace FOG
 
             if (cert == null) return;
 
             try
             {
-                var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
+                var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
                 store.Open(OpenFlags.ReadWrite);
                 store.Remove(cert);
                 store.Close();
             }
             catch (Exception ex)

Unless I’m missing something, those are the changes you made, yes? I am a bit confused about why it is trying to throw it in /usr/share. There has been a change in the version of mono (6.0.0.327), but running the version of SmartInstaller.exe you sent works as expected when executed under the context of root.

There anything else I should need to clean out other than SetupHelper.* in ./bin to make sure it has the right version in the exe build? Other thoughts?

Sebastian Roth

@Daniel-Miller said in Trouble installing SmartClient on MacOS Sierra:

Injecting root CA

Obviously we have missed RSA.cs in Zazzles! There are two places in that code file, line 155 and line 202.

Zazzles is another github repo you need to clone and open in Visual Studio. I’d suggest you don’t use the latest master but checkout commit 0f13a4caf as I have messed with the external libraries here as well. After it’s built you need to manually copy the new Zazzles.dll over to your fog-client\libs (from the top of my head I think this is the correct location) folder, close Zazzles project in Visual Studio, reopen fog-client project and re-build the solution.

We are getting there!

If for some reason FOGService is not properly starting or you see some very strange behaviour, then let me know. Don’t waste your time to try and figure it out. It is possible that I got something wrong with the hints here - writing this while sitting in the train and don’t have all my tools here.

Daniel Miller

@Sebastian-Roth Alrightly, rebuilt Zazzles with the change and now the installer is happy. FOGService isn’t starting, however, or at least fog tray isn’t starting and when I manually start it it is complaining about not being able to connect to the bus.

Sebastian Roth

@Daniel-Miller said in Trouble installing SmartClient on MacOS Sierra:

FOGService isn’t starting, however, or at least fog tray isn’t starting and when I manually start it it is complaining about not being able to connect to the bus.

Can you post more details on this? Log output?

Daniel Miller

@Sebastian-Roth
/opt/fog-service/fog.log was an empty file after rebooting the machine. Manually launching the service (sudo /opt/fog-service/control.sh start) produces the familiar logs but complains about FOG Server CA NOT found in keystore:

 9/17/2019 10:09 AM Main Overriding exception handling
 9/17/2019 10:09 AM Main Bootstrapping Zazzles
 9/17/2019 10:09 AM Controller Initialize
 9/17/2019 10:09 AM Controller Start

 9/17/2019 10:09 AM Service Starting service
 9/17/2019 10:09 AM Bus Became bus server
 9/17/2019 10:09 AM Bus Emmiting message on channel: Status
 9/17/2019 10:09 AM Service Invoking early JIT compilation on needed binaries

------------------------------------------------------------------------------
--------------------------------Authentication--------------------------------
------------------------------------------------------------------------------
 9/17/2019 10:09 AM Client-Info Version: 0.11.16
 9/17/2019 10:09 AM Client-Info OS:      Mac
 9/17/2019 10:09 AM Middleware::Authentication Waiting for authentication timeout to pass
 9/17/2019 10:09 AM Middleware::Communication Download: http://10.10.25.15/fog/management/other/ssl/srvpublic.crt
 9/17/2019 10:09 AM Data::RSA ERROR: Unable to retrieve FOG Server CA
 9/17/2019 10:09 AM Data::RSA ERROR: FOG Server CA NOT found in keystore
 9/17/2019 10:09 AM Middleware::Authentication ERROR: Could not authenticate
 9/17/2019 10:09 AM Middleware::Authentication ERROR: Value cannot be null.
Parameter name: authority

Manually relaunching the service under the root user context (sudo -i /opt/fog-service/control.sh start) shows it starting to communicate normally:

------------------------------------------------------------------------------
--------------------------------Authentication--------------------------------
------------------------------------------------------------------------------
 9/17/2019 10:15 AM Client-Info Version: 0.11.16
 9/17/2019 10:15 AM Client-Info OS:      Mac
 9/17/2019 10:15 AM Middleware::Authentication Waiting for authentication timeout to pass
 9/17/2019 10:17 AM Controller Stop
 9/17/2019 10:17 AM Service Stop requested
 9/17/2019 10:17 AM Middleware::Authentication ERROR: Could not authenticate
 9/17/2019 10:17 AM Middleware::Authentication ERROR: Thread was being aborted.
 9/17/2019 10:17 AM Bus Emmiting message on channel: Status
 9/17/2019 10:17 AM Main Overriding exception handling
 9/17/2019 10:17 AM Main Bootstrapping Zazzles
 9/17/2019 10:17 AM Controller Initialize
 9/17/2019 10:17 AM Controller Start

 9/17/2019 10:17 AM Service Starting service
 9/17/2019 10:17 AM Bus Became bus server
 9/17/2019 10:17 AM Bus Emmiting message on channel: Status
 9/17/2019 10:17 AM Service Invoking early JIT compilation on needed binaries

------------------------------------------------------------------------------
--------------------------------Authentication--------------------------------
------------------------------------------------------------------------------
 9/17/2019 10:17 AM Client-Info Version: 0.11.16
 9/17/2019 10:17 AM Client-Info OS:      Mac
 9/17/2019 10:17 AM Middleware::Authentication Waiting for authentication timeout to pass
 9/17/2019 10:17 AM Middleware::Communication Download: http://10.10.25.15/fog/management/other/ssl/srvpublic.crt
 9/17/2019 10:17 AM Data::RSA FOG Server CA cert found
 9/17/2019 10:17 AM Middleware::Authentication Cert OK
 9/17/2019 10:17 AM Middleware::Authentication No token found at /opt/fog-service/token.dat, this is expected if the client has not authenticated before
 9/17/2019 10:17 AM Middleware::Authentication ERROR: Could not get security token
 9/17/2019 10:17 AM Middleware::Authentication ERROR: Could not find file "/opt/fog-service/token.dat"
 9/17/2019 10:17 AM Middleware::Communication POST URL: http://fogserver/fog/management/index.php?sub=requestClientInfo&authorize&newService
 9/17/2019 10:17 AM Middleware::Response Invalid host
 9/17/2019 10:17 AM Middleware::Communication URL: http://fogserver/fog/service/register.php?hostname=ASCAS-TestImac.local&mac=38:C9:86:4F:EE:1E|28:F0:76:49:AD:70|0A:F0:76:49:AD:70|26:EA:E7:94:F6:3A|1A:00:01:E3:00:70|1A:00:01:E3:00:71|1A:00:01:E3:00:70&newService&json

So, it looks like the service works, but the daemon and user agent aren’t loaded correctly. I actually don’t see where those service loads happen during install. I would expect they would be handled in Install() in MacInstall.cs as Uninstall() calls unload for the services, but the only reference I see to the load call are in StartService() in MacUpdate.cs and the OnUpdate event handler in FogUserService.cs.

Trouble installing SmartClient on MacOS Sierra

222

11.6k

17.1k

154.6k