Snapins dont deploy from Storage Nodes. Hash Mismatch

[Greg Plamondon]

I am trying to deploy snapins from our Storage nodes and I am getting an error in the fog.log.

------------------------------------------------------------------------------
---------------------------------SnapinClient---------------------------------
------------------------------------------------------------------------------
 7/21/2016 1:40 PM Client-Info Client Version: 0.11.4
 7/21/2016 1:40 PM Client-Info Client OS:      Windows
 7/21/2016 1:40 PM Client-Info Server Version: 1.3.0-RC-1
 7/21/2016 1:40 PM Middleware::Response Success
 7/21/2016 1:40 PM SnapinClient Snapin Found:
 7/21/2016 1:40 PM SnapinClient     ID: 378
 7/21/2016 1:40 PM SnapinClient     Name: 01 - TeamViewer
 7/21/2016 1:40 PM SnapinClient     Created: 2016-07-21 13:39:35
 7/21/2016 1:40 PM SnapinClient     Action:
 7/21/2016 1:40 PM SnapinClient     Pack: False
 7/21/2016 1:40 PM SnapinClient     Hide: False
 7/21/2016 1:40 PM SnapinClient     Server: http://192.168.18.39/fog
 7/21/2016 1:40 PM SnapinClient     TimeOut: 0
 7/21/2016 1:40 PM SnapinClient     RunWith: cmd.exe
 7/21/2016 1:40 PM SnapinClient     RunWithArgs: /c
 7/21/2016 1:40 PM SnapinClient     Args:
 7/21/2016 1:40 PM SnapinClient     File: TeamViewer-AllSites.sfx.exe
 7/21/2016 1:40 PM Middleware::Communication Download: http://192.168.18.39/fog/service/snapins.file.php?mac=C0:3F:D5:9C:6C:36||00:00:00:00:00:00:00:E0&taskid=378
 7/21/2016 1:40 PM SnapinClient C:\Program Files (x86)\FOG\tmp\TeamViewer-AllSites.sfx.exe
 7/21/2016 1:40 PM SnapinClient ERROR: Hash does not match
 7/21/2016 1:40 PM SnapinClient ERROR: --> Ideal: 9B87981143F24BF592603402DE7D6F5E8B6BDCC97F37328C8474DB95635C07CAF792B8ECCE422A6235AE3F130E717417B902059580C62314E4026BBEC9889E4B
 7/21/2016 1:40 PM SnapinClient ERROR: --> Actual: 7FF6B7E4378EC39EFFC8A042F445AA6618E765B6F39C8BC53FA5F1E3A5924129F239A6CA0FD7C0350D63C808EFB2FD3E18D48EE6A151C75CCA4C3D1E0B6A2A23
 7/21/2016 1:40 PM Middleware::Communication URL: http://10fogserver/fog/service/snapins.checkin.php?taskid=378&exitcode=-1&mac=C0:3F:D5:9C:6C:36||00:00:00:00:00:00:00:E0&newService&json
------------------------------------------------------------------------------

Although: http://18fogserver/fog/status/getsnapinhash.php?filepath=/opt/fog/snapins/TeamViewer-AllSites.sfx.exe Returns

9b87981143f24bf592603402de7d6f5e8b6bdcc97f37328c8474db95635c07caf792b8ecce422a6235ae3f130e717417b902059580c62314e4026bbec9889e4b|10239200

Remote Node

[root@18fogserver snapins]# sha512sum TeamViewer-AllSites.sfx.exe
9b87981143f24bf592603402de7d6f5e8b6bdcc97f37328c8474db95635c07caf792b8ecce422a6235ae3f130e717417b902059580c62314e4026bbec9889e4b  TeamViewer-AllSites.sfx.exe

Our Main Fogserver

root@10fogserver:/opt/fog/snapins $ sha512sum TeamViewer-AllSites.sfx.exe
9b87981143f24bf592603402de7d6f5e8b6bdcc97f37328c8474db95635c07caf792b8ecce422a6235ae3f130e717417b902059580c62314e4026bbec9889e4b  TeamViewer-AllSites.sfx.exe

Thanks Tom for helping me troubleshoot this.

[Tom Elliott]

Also, to add further information:

Downloading the file using the URL returns the proper file and it is indeed usable. It also works using the “client” url path as it used to.

So the ‘location’ url is being used (from what we can see in the file information), but it’s not returning the proper hash if it’s from the location.

[Greg Plamondon]

Hey guys any progress on this problem? Am I the only one that uses the nodes to install snapins?

[Wayne Workman]

@Greg-Plamondon said in Snapins dont deploy from Storage Nodes. Hash Mismatch:

Am I the only one that uses the nodes to install snapins?

No.
We need to use nodes, too.

[Tom Elliott]

I haven’t had any time to validate this. I know “of” the issue, but no idea how to fix it.

[Wayne Workman]

@Tom-Elliott So previously, snapins were pulled from the main.

Now, with locations, they are pulled from the nodes and this basically blocks deploying snapins completely at any location besides the main location.

[Tom Elliott]

@Wayne-Workman No, snapins were not being pulled from the main. The main was sending the file, but the file was being downloaded from the location. It was a “middle man” if you will.

Now the files are downloading, but something’s not allowing the file to download fully, or the hash is being calculated improperly.

[Wayne Workman]

@Tom-Elliott Still yet, the point is we can’t deploy any snapins to anywhere except the main location if locations are setup.

[Wayne Workman]

Moving this to bugs, since people are unable to deploy some or all snapins to remote locations while locations are setup.

[Greg Plamondon]

@Wayne-Workman
I fould that if you change the location to the main location after Imaging the snapins will deploy from the main site. So basically you do an Image without snapins. once its imaged, change the location to the main location. then advanced task “All-Snapins” Its a workaround we are using currently.

[Tom Elliott]

Based on our finding this morning, I’m leaning there’s more likely a networking issue going on.

@Greg-Plamondon can you please see if this happens with ALL nodes and ALL snapins? I know that may be asking a lot, but we found out that the issue’s you’re experiencing maybe just your environment. There are other potentials as well, and that’s not to say there isn’t necessarily a bug here. However, it seems it’s more likely related to a potential networking issue now.

For that reason, I’ve moved this out of bugs and back into FOG Problems. Joe and I still want to help figure this out, just want to first ensure this is simply not a bad patch cable, or some other firewall filters causing issues.

[Greg Plamondon]

@Tom-Elliott

I am not sure what kind of network issue would cause a
Hash mismatch with the exact same hash values every time? If you guys know what kind of network issues I should be looking at I will investigate further. I don’t have any firewalls in between nodes and Master server, The iptables is shutdown and disabled on nodes and master. This issue happens to me on all 5 remote nodes that are in different locations connected via MPLS.

Thanks.,

[Wayne Workman]

@Greg-Plamondon I had the same issue you had. Turned out, a fiber line that supported our entire VM platform was literally cut and left laying for a week by a contractor. All VM Platform traffic was going across a 100 meg copper link. Once that was fixed, everything immediately started working perfectly.

I’m happy to try to help diagnose what’s going on via team viewer. I don’t know if you’re still online or not, this sort of troubleshooting may take time and I can’t do it while at work.

[Greg Plamondon]

@Wayne-Workman said in Snapins dont deploy from Storage Nodes. Hash Mismatch:

a fiber line tha

Thanks Wayne, If your available and willing some time this weekend. let me know.

[Wayne Workman]

@Greg-Plamondon Hey buddy I’ll be at my laptop most of the day. Message me, and I should get back to you sometime soonish.

[Wayne Workman]

@Greg-Plamondon Any word on this?

[Tom Elliott]

Fairly sure this is fixed now. I don’t know why the hashes were all sorts of strange, but I believe this was partially related to the timeout as seen https://forums.fogproject.org/topic/8278/rc6-snapins-no-longer-working/13.

With any luck, this was all it was, and I’m just going to guess what was seen from the start was from a download that timed timed out, while the hashing mechanism was able to run much faster and return within its own allotted time.