Disable snapin hashing

JJ Fullmer

Would like to disable snapin hashing

The way I use snapins, I have one script that takes arguments. The script essentially makes either a interactive or background scheduled task to install a chocolatey package given in the parameters to the script. The script is simple and it works, and really I don’t even need it included with the snapins because I have it installable from an internal powershell repo and could just have the snapin tool run a powershell command. But a snapin currently requires a file, and that file has a hash, and checking it seems to be required. Somehow my hash keeps changing, so I want to disable the hash checking.

The scenario in more detail

So all my snapins use the same powershell script. This script hasn’t changed in some time. However, somehow the hash keeps changing when I try to deploy snapins. I have some automations in place using the api that make my snapins for me and I’ve tried many things to try and ensure the hash is right. However I have found that if I try to create a new snapin with a ‘snapin file exists’ setting it gets the wrong hash. Or if I do it with the api, the has doesn’t match. I even make it so every time I push a snapin through the api it re-uploads the powershell script, grabs the hash of the uploaded script and updates the hash value in the database for every snapin. But it still doesn’t stick 100%.

Easiest solution

While it is possible that there’s something I’m doing somewhere that keeps causing the hash to change. I would find it much easier for my environment to disable the hashing. The hash check is a cool feature, and I remember when it was being added for cases where people have large files they’re sending in a snapin and want to make sure the whole file downloaded. But I’m sending one tiny script file of a few kb that doesn’t change. Since it’s somehow changing, I’d like the option to just disable that feature for my purposes. So I’m wondering if there’s any easy way that’s built in that I don’t know about. Or if I have to go into the database and change something, or some setting file on the server or if I have to recompile the client installer to have a custom client that doesn’t do snapin hashes. I’m willing to do what it takes, just need a little help.

Environment

Fog Server 1.5.5
Cent OS 7 x64
Client version 0.11.16

Please and thank you

JJ Fullmer

Here is the code I use to create a snapin after publishing a chocolatey package to my repo.
I added the hashes after the problem started and it sometimes helps but it seems the behavior is slightly unpredictable and the hash record on fog still changes somehow.


        Write-Verbose "making sure package $global:packageName exists as fog snapin";
        if ( (Invoke-FogApi -uriPath "snapin/search/$global:packageName" -Method GET).count -eq 0){
            Write-Verbose "snapin does not exist, creating new snapin";
            $snapinScript = Get-Item "path\to\chocoPkgSnapin.ps1";
            $hash = ($snapinScript | Get-FileHash -Algorithm SHA512).Hash;
            $fileSize = $snapinScript.Length;
            $json = @{
                "name"="$global:packageName"
                "file"="chocoPkgSnapin.ps1"
                "args"="-pkgname $global:packageName"
                "reboot"=""
                "shutdown"=""
                "runwith"="powershell.exe"
                "runwithArgs"="-ExecutionPolicy Bypass -NoProfile -File"
                "protected"="0"
                "isEnabled"="1"
                "toReplicate"="1"
                "hide"="0"
                "timeout"="0"
                "packtype"="0"
                "storagegroupname"="default"
                "hash"="$hash"
                "size"="$fileSize"
            } | ConvertTo-Json;
            Invoke-FogApiChocoSnapin -uriPath 'snapin/new' -Method POST -jsonData $json -verbose;
        }
        else {
            Write-Verbose "Snapin already exists";
        }
        Write-Verbose "Updating hases for all snapins";
        $snapinScript = Get-Item "path\to\chocoPkgSnapin.ps1";
        $hash = ($snapinScript | Get-FileHash -Algorithm SHA512).Hash;
        $fileSize = $snapinScript.Length;
        $snapins = Get-FogObject -type object -coreObject snapin;
        $snapins.snapins | Where-Object file -match 'choco' | ForEach-Object { $data = @{
            "id" = "$($_.id)";
            "name" = "$($_.name)";
            "file" = "$($_.file)";
            "runwith"="powershell.exe";
            "runwithArgs"="-ExecutionPolicy Bypass -NoProfile -File";
            "args" = "$($_.args)";
            "protected"="0";
            "isEnabled"="1";
            "toReplicate"="1";
            "hide"="0";
            "timeout"="0";
            "packtype"="0";
            "reboot"="";
            "shutdown"="";
            "size"="$fileSize"
            "hash" = "$hash";
        } | convertto-json; 
        Update-FogObject -type object -coreObject 'snapin' -IDofObject $_.id  -jsonData $data -uri "snapin/$($_.ID)/
        Write-Verbose 'Done!';
        return;

Some of the snapins return 500 errors when I attempt to loop through them all and update their hash records.
Since that isn’t working I’m really hoping there’s some way to disable the hashing function, even if it’s some hackish way in the database or something.

Sebastian Roth

@JJ-Fullmer I found some time to dive into this and figured that I am still missing the initial question/answer. What is actually causing an issue for you here? Sorry if this sounds like a dumb question but I am still not sure how to try and replicate the issue you see as I can’t see the exact error it is causing for you!?

Do you see a message in the fog-client log like this?

Hash does not match
--> Ideal: ...
--> Actual: ...

Or is it the snapin replication that causes you grief?

Testing the script you posted I ran into an error:

...
        Write-Verbose "Updating hases for all snapins";
        $snapinScript = Get-Item Get-Item "path\to\chocoPkgSnapin.ps1";

Get-Item called on Get-Item fails for me. I suppose this was just a copy&paste thin.

While I understand that your chocoPkgSnapin.ps1 script has not changed in some time I still wonder if you are aware of your script code seems to not actually upload the file contents. While it does create a snapin definition it does not update the chocoPkgSnapin.ps1 on the server at all. I might be wrong with that. You know I am more a Linux shell than MS Powershell guy. But I am sure we’ll figure this out.

Disabling hashing is not something I want to rule out. Don’t get me wrong on this. I am just trying to get my head around this before I can see what we are heading for with this.

Sebastian Roth

@JJ-Fullmer said in Disable snapin hashing:

Invoke-FogApiChocoSnapin

Where is this defined?

JJ Fullmer

@Sebastian-Roth That is a internal function in that script. I’m pretty sure that it is the same as the Invoke-FogApi function I have in the published powershell module, so you could find it here https://github.com/FOGProject/fog-community-scripts/blob/master/PowershellModules/FogApi/FogApi.psm1
It’s renamed here to avoid clobberring I think as I may have written the code I pasted as an example here before I made the uninversal module.

JJ Fullmer

@Sebastian-Roth That is the general idea of what happens in the fog log yes. I get a hash doesn’t match error.
I took out the paths when I copied pasted the script, I think that duplicate get-item is just a typo.
I beleive I am trying to not update the file on the server here, instead of trying to re-upload the file everytime I am just getting the correct information about the file (name, size, hash) and trying to force every snapin database entry to match the file’s details correctly. Somehow though I end up with different hash’s during deployment. If I export the snapins, they all look the same in the exported csv as they should. It’s like something happens when the snapin is deployed where it changes.

JJ Fullmer

@Sebastian-Roth

function Invoke-FogApiChocoSnapin {
<#
    .SYNOPSIS
        a cmdlet function for making fogAPI calls via powershell

    .DESCRIPTION
        takes a few parameters with a default that will get all hosts
        Makes a call to the api of a fog server and returns the results of the call
        The returned value is an object that can then be easily filtered, processed, and otherwise manipulated in poweshell.
        i.e. you could take the return value of the default all hosts and run
            $(invoke-fogapiChocoSnapin).hosts | where name -match "$(hostname)"
        to get the host information for the current computer

    .PARAMETER fogApiToken
        a string of your fogApiToken gotten from the fog web ui. Can be set in the function as a default or passed to the function

    .PARAMETER fogUserToken
        a string of your fog user token gotten from the fog web ui in the user section. Can be set in the function as a default or passed to the function

    .PARAMETER fogServer
        The hostname or ip address of your fogserver, defaults to the default fog-server

    .PARAMETER uriPath
        Put in the path of the apicall that would follow http://fog-server/fog/
        i.e. 'host/1234' would access the host with an id of 1234

    .PARAMETER Method
        Defaults to 'Get' can also be

    .PARAMETER jsonData
        The jsondata string for including data in the body of a request

    .EXAMPLE
        #if you had the api tokens set as default values and wanted to get all hosts and info you could run this, assuming your fogserver is accessible on http://fog-server
        Invoke-FogApiChocoSnapin;

    .Example
        #if your fogserver was named rawr and you wanted to put rename host 123 to meow
        Invoke-FogApiChocoSnapin -fogServer "rawr" -uriPath "host/123" -Method "Put" -jsonData "{ `"name`": meow }";

    .Link
        https://news.fogproject.org/simplified-api-documentation/

    .NOTES
        The online version of this help takes you to the fog project api help page

#>

    [CmdletBinding()]
    param (
        [string]$fogApiToken = '',
        [string]$fogUserToken = '',
        [string]$fogServer = "fog-server",
        [string]$uriPath = "host", #default to get all hosts
        [string]$Method = "Get",
        [string]$jsonData #default to empty
    )

    begin {
        # Create headers
        Write-Verbose "Building Headers...";
        $headers = @{};
        $headers.Add('fog-api-token', $fogApiToken);
        $headers.Add('fog-user-token', $fogUserToken);

        # Set the baseUri
        Write-Verbose "Building api call URI...";
        $baseUri = "http://$fogServer/fog";
        $uri = "$baseUri/$uriPath";

    }

    process {

        Write-Verbose "$Method`ing $jsonData to/from $uri";
        if ($Method -eq "Get") {
            $result = Invoke-RestMethod -Uri $uri -Method $Method -Headers $headers -ContentType "application/json";
        }
        else {
            $result = Invoke-RestMethod -Uri $uri -Method $Method -Headers $headers -Body $jsonData -ContentType "application/json";
        }
    }

    end {
        Write-Verbose "finished api call";
        return $result;
    }
}

Sebastian Roth

@JJ-Fullmer said in Disable snapin hashing:

That is the general idea of what happens in the fog log yes. I get a hash doesn’t match error.

Ok, in this case we’d need to provide a new patched fog-client installer binary for you that doesn’t do the hash sum checking or has an option to select if it should check or not. But you’d need to deploy the new client installer to all your hosts. I am wondering if that’s all worth it. Probably easier to figure out why the hash mismatch is being caused while it shouldn’t be?!

First let’s try to gather some more information. Leave the snapin as is for now and only schedule snapin tasks for clients and let those run.

Does it randomly fail on the same host? So if you schedule the same task on one host ten times in a row, does it run through all the time if it is fine on the first run?
When it magically fails on a host can you please grab that host and take a look at C:\Program Files (x86)\FOG\tmp. I have not checked the code yet but I would hope the fog-client leaves a copy of that file there on failure. Now check if that file is zero size or truncated or anything like that.
On snapin failure as well pay attention to the apache and PHP-FPM log files to see if there is an HTTP/PHP error on snapin download (see my signature).
Please post the full fog-client log here in case of a failure so we can have a look. Sometimes there is something we see that leads us to the real problem.

If that doesn’t help we can go ahead an try monitor the database for changes of the hash. FOG has a deamon running that updates the snapin hashes from time to time. Unlikely but maybe this is causing an issue. You can take a look in that log file (webUI -> logs -> snapin log) and even disable that in another step just to see if it interferes.

JJ Fullmer

First let’s try to gather some more information. Leave the snapin as is for now and only schedule snapin tasks for clients and let those run.

Does it randomly fail on the same host? So if you schedule the same task on one host ten times in a row, does it run through all the time if it is fine on the first run?

I don’t typically try it more than once maybe twice, if it fails I just run the command manually. But I will give that a try. See if I can’t recreate it on my computer.

When it magically fails on a host can you please grab that host and take a look at C:\Program Files (x86)\FOG\tmp. I have not checked the code yet but I would hope the fog-client leaves a copy of that file there on failure. Now check if that file is zero size or truncated or anything like that.

It does not leave a copy, it does put a copy there and if you’re watching it you can copy it real quick or you can stop the fogservice before the hash check if you’re lucky. I think I did that once when I first discovered the issue but don’t recall the result, I remember it being confusing, I guess I’ll have to try this again too.

On snapin failure as well pay attention to the apache and PHP-FPM log files to see if there is an HTTP/PHP error on snapin download (see my signature).

Good thought will give those a post.

Please post the full fog-client log here in case of a failure so we can have a look. Sometimes there is something we see that leads us to the real problem.

Standby

If that doesn’t help we can go ahead an try monitor the database for changes of the hash. FOG has a deamon running that updates the snapin hashes from time to time. Unlikely but maybe this is causing an issue. You can take a look in that log file (webUI -> logs -> snapin log) and even disable that in another step just to see if it interferes.

I was hoping there is a way to disable it on the server, maybe something in the database or something, because I didn’t want to go through the trouble of recompiling the fog client. The other trick is that I really only use snapins during provisioning right after I image a computer. But I guess if I want it fixed I oughta suck it up.

The only other thing that may be a factor is I also add the snapins to the host and initiate the task from the api during provisioning. So It’s possible that it being queued from api causes this too, we’ll see if that’s the case though since I’ll be testing by deploying through the gui.

JJ Fullmer

Ok so it appears to be working when I queued the tasks from the gui. I did grab the file while something was deploying and the hash was the same this time.
So maybe this only happens when the snapin is queued from the api, doesn’t make a ton of sense, but maybe.

Could also be related to when a host is new perhaps. I will try to be more vigilante in logging the next time it happens if I can’t recreated it by queuing the snapins from the api

ps this would be how I am queuing a all snapins task in powershell

function Start-FogSnapins {
    [CmdletBinding()]
    param (
        $hostid = ((Get-FogHost).id),
        $taskTypeid = 12
    )

    begin {
        Write-Verbose "Stopping any queued snapin tasks";
        $tasks = Invoke-FogApi -Method GET -uriPath "task/active";
        $taskID = (($tasks | Where-Object hostID -match $hostid).id);
        Write-Verbose "Found $($taskID.count) tasks deleting them now";
        $taskID | ForEach-Object{
            Invoke-FogApi -Method DELETE -uriPath "task/$_/cancel";
        }
        # $snapAssocs = Invoke-FogApi -uriPath snapinassociation -Method Get;
        # $snaps = $snapAssocs.snapinassociations | ? hostid -eq $hostid;
    }

    process {
        Write-Verbose "starting all snapin task for host";
        $json = (@{
            "taskTypeID"=$taskTypeid;
            "deploySnapins"=-1;
        } | ConvertTo-Json);
        Invoke-FogApi -Method POST -uriPath "host/$hostid/task" -jsonData $json;
    }

    end {
        Write-Verbose "Snapin tasks have been queued on the server";
        return;
    }
}

function Get-FogHost {
    [CmdletBinding()]
    param (
        [string]$uuid,
        [string]$hostName,
        [string]$macAddr
    )

    begin {
        [bool]$found = $false;
        Write-Verbose 'Checking for passed variables'
        if (!$uuid -and !$hostName -and !$macAddr) {
            Write-Verbose 'no params given, getting current computer variables';
            $uuid = (Get-WmiObject Win32_ComputerSystemProduct).UUID;
            $macAddr = ((Get-NetAdapter | Select-Object MacAddress)[0].MacAddress).Replace('-',':');
            $hostName = $(hostname);
        }
        Write-Verbose 'getting all hosts to search...';
        $hosts = (Invoke-FogApi).hosts;
        Write-Verbose "search terms: uuid is $uuid, macAddr is $macAddr, hostname is $hostName";
    }

    process {
        Write-Verbose 'finding host in hosts';
        [bool]$found = $false;
        $hostObj = $hosts | Where-Object {
            ($uuid -ne "" -AND $_.inventory.sysuuid -eq $uuid) -OR `
            ($hostName -ne "" -AND $_.name -eq $hostName) -OR `
            ($macAddr -ne "" -AND $_.macs -contains $macAddr);
            if  ($uuid -ne "" -AND $_.inventory.sysuuid -eq $uuid) {
                 Write-Verbose "$($_.inventory.sysuuid) matches the uuid $uuid`! host found";
                 $found = $true;
            }
            if ($macAddr -ne "" -AND $_.macs -contains $macAddr) {
                Write-Verbose "$($_.macs) matches the macaddress $macAddr`! host found";
                $found = $true;
            }
            if  ($hostName -ne "" -AND $_.name -eq $hostName) {
                Write-Verbose "$($_.name) matches the hostname $hostName`! host found";
                $found = $true;
            }
        }

    }

    end {
        if ($found){
            return $hostObj;
        }
        return $found; #return false if host not found
    }
}

JJ Fullmer

I can’t seem to recreate it, but I know it’s broken whenever I deploy a new computer. So next time I have to image a computer I’ll come on back

mparlette

@JJ-Fullmer @Sebastian-Roth I would really like some help in this area as well. I am creating snapins and just got stuck in the same spot.

---------------------------------SnapinClient---------------------------------

4/23/2019 12:39 PM Client-Info Client Version: 0.11.16
4/23/2019 12:39 PM Client-Info Client OS: Windows
4/23/2019 12:39 PM Client-Info Server Version: 1.5.4
4/23/2019 12:39 PM Middleware::Response Success
4/23/2019 12:39 PM SnapinClient Running snapin test
4/23/2019 12:39 PM SnapinClient ERROR: Snapin hash does not exist

@JJ-Fullmer check it out using your stuff
$dataSet = @{

“name” = “test4”
“description” = “test4”
“file” = “MA-Redirect.ps1”
“args” = “747EPO.CDA.BEA.LAB”
“runWith” = “powershell.exe”
“isEnabled” = “1”
“runWithArgs” = “-ExecutionPolicy Bypass -NoProfile -File”

}

$dataToSend = ($dataSet | ConvertTo-JSON)

New-FogObject -type object -coreObject snapin -jsonData $dataToSend

Sebastian Roth

@mparlette said in Disable snapin hashing:

SnapinClient ERROR: Snapin hash does not exist

Is the FOGSnapinHash service running on your FOG server? How long did you wait after clearing the snapin?

mparlette

@Sebastian-Roth Yes.

My setup is almost exact to @JJ-Fullmer except I am on 1.5.4

[root@localhost snapins]# systemctl status FOGSnapinHash
● FOGSnapinHash.service - FOGSnapinHash
Loaded: loaded (/usr/lib/systemd/system/FOGSnapinHash.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2019-02-05 16:19:42 PST; 2 months 15 days ago
Main PID: 2447 (FOGSnapinHash)
CGroup: /system.slice/FOGSnapinHash.service
├─2447 /usr/bin/php -q /opt/fog/service/FOGSnapinHash/FOGSnapinHash &
└─2475 /usr/bin/php -q /opt/fog/service/FOGSnapinHash/FOGSnapinHash &

Feb 05 16:19:42 localhost systemd[1]: Started FOGSnapinHash.
Feb 05 16:19:42 localhost systemd[1]: Starting FOGSnapinHash…

mparlette

@Sebastian-Roth OK. I just didn’t wait long enough sorry to rattle your cage. Thanks for the help it did the hash just waited a while and it’s working now.

Disable snapin hashing

Would like to disable snapin hashing

The scenario in more detail

Easiest solution

Environment

---------------------------------SnapinClient---------------------------------

200

12.0k

17.3k

155.2k