Snapins blocked and remain in "Checked in"

Seydoo

Server

FOG Version: 1.3.5
OS: CentOs 7

Client

Service Version: 0.11.11
OS: W.10

Description

Hello,

I just install the last release of Fog on a new server. I was in 1.3 RC15 before (initially in 1.2) on RedHat 6.5 and prefered for much reasons to have a clean install.
No problem with install (just had to install php 7 instead of 5.xx already present in Centos), My host were well imported (automatically in the web gui, just have to approve MAC) with old client, and I recreated all my snapins to be sure for the deployment (I had some trouble with 1.3 : some snapins downloaded well but never install from /fog/tmp (svn, skype, oracle) )
But and it is why i created this post, none of the snapins are deploying and stay blocked in “Checked In” status. I also have this error in fog.log : “object reference is not set to an instance of an object”
Reset encryption data is not available on the host page so I tried create a group to have this functionality, i have it but nothing change after reset

Edit :
here is my fog.log

25/04/2017 12:21 Client-Info Client Version: 0.11.11
 25/04/2017 12:21 Client-Info Client OS:      Windows
 25/04/2017 12:21 Client-Info Server Version: 1.3.5
 25/04/2017 12:21 Middleware::Response ERROR: Unable to get subsection
 25/04/2017 12:21 Middleware::Response ERROR: La référence d'objet n'est pas définie à une instance d'un objet.
 25/04/2017 12:21 Service Sleeping for 99 seconds
 25/04/2017 12:23 Middleware::Communication URL: http://172.27.1.1/fog/management/index.php?sub=requestClientInfo&configure&newService&json
 25/04/2017 12:23 Middleware::Response Success
 25/04/2017 12:23 Middleware::Communication URL: http://172.27.1.1/fog/management/index.php?sub=requestClientInfo&mac=E8:40:F2:97:80:55||00:00:00:00:00:00:00:E0|00:00:00:00:00:00:00:E0&newService&json
 25/04/2017 12:23 Middleware::Authentication Waiting for authentication timeout to pass
 25/04/2017 12:23 Middleware::Communication Download: http://172.27.1.1/fog/management/other/ssl/srvpublic.crt
 25/04/2017 12:23 Data::RSA FOG Server CA cert found
 25/04/2017 12:23 Middleware::Authentication Cert OK
 25/04/2017 12:23 Middleware::Communication POST URL: http://172.27.1.1/fog/management/index.php?sub=requestClientInfo&authorize&newService
 25/04/2017 12:23 Middleware::Response Private key not found
 25/04/2017 12:23 Middleware::Response Success
 25/04/2017 12:23 Middleware::Communication URL: http://172.27.1.1/fog/service/getversion.php?clientver&newService&json
 25/04/2017 12:23 Middleware::Communication URL: http://172.27.1.1/fog/service/getversion.php?newService&json

 25/04/2017 12:23 Service Creating user agent cache
 25/04/2017 12:23 Middleware::Response ERROR: Unable to get subsection
 25/04/2017 12:23 Middleware::Response ERROR: La référence d'objet n'est pas définie à une instance d'un objet ***(in english : object reference is not set to an instance of an object)***
 25/04/2017 12:23 Middleware::Response ERROR: Unable to get subsection
 25/04/2017 12:23 Middleware::Response ERROR: La référence d'objet n'est pas définie à une instance d'un objet.
 25/04/2017 12:23 Middleware::Response ERROR: Unable to get subsection
 25/04/2017 12:23 Middleware::Response ERROR: La référence d'objet n'est pas définie à une instance d'un objet.

Tom Elliott

@Seydoo said in Snapins blocked and remain in "Checked in":

25/04/2017 12:23 Middleware::Response Private key not found

This is likely the problem.

Assuming you still have the original fog server, you might want to copy the original server’s /opt/fog/snapins directory ENTIRELY to the new server you created.

Seydoo

Unfortunately the problem is still present after copy - paste /opt/fog/snapins directory

Joe Schmitt

@Seydoo did you follow these steps: https://wiki.fogproject.org/wiki/index.php?title=FOG_Client#Maintain_Control_Of_Hosts_When_Building_New_Server ?

If you have completey copied over the snapins directory (including hidden files), then re-run the fog installer on the new server with the --recreate-keys flag present (this is all described in the wiki article).

Seydoo

@Joe-Schmitt
I followed all steps in the wiki but I still have the private key not found in fog.log
something wrong in my .fogsetting ?

#Start of FOG Settings
#Created by the FOG Installer
#Find more information about this file in the FOG Project wiki:
#https://wiki.fogproject.org/wiki/index.php?title=.fogsettings
#Version: 1.3.5
#Install time: ven. 14 avril 2017 13:33:09 CEST
ipaddress='172.27.1.1'
copybackold='0'
interface='ens192'
submask='255.255.255.0'
routeraddress='172.27.1.1'
plainrouter='172.27.1.1'
dnsaddress='172.16.0.14'
username='fog'
password="3MHirm59hScsHbOnaLPNIcvfZKY/HqEd131dcHETI48="
osid='1'
osname='Redhat'
dodhcp='Y'
bldhcp='1'
dhcpd='dhcpd'
blexports='1'
installtype='N'
snmysqluser='root'
snmysqlpass=''
snmysqlhost='localhost'
installlang=''
donate='0'
storageLocation='/meti/images'
fogupdateloaded=1
docroot='/var/www/html/'
webroot='/fog/'
caCreated='yes'
startrange='172.27.1.10'
endrange='172.27.1.254'
bootfilename='undionly.kpxe'
packages='bc curl dhcp gcc gcc-c++ gettext gzip httpd lftp m4 make mariadb mariadb-server mod_ssl net-tools nfs-utils php php-bcmath php-cli php-common php-fpm php-gd php-ldap php-mbstring php-mysqlnd php-process tar tftp-server unzip vsftpd wget xinetd'
noTftpBuild=''
notpxedefaultfile=''
sslpath='/opt/fog/snapins/ssl/'
backupPath=''
php_ver=''
php_verAdds=''
sslprivkey='/opt/fog/snapins/ssl/.srvprivate.key'
#End of FOG Settings

Joe Schmitt

@Seydoo did you rerun the installer with --recreate-keys?

Seydoo

yes I did
I reinstall the fog client on my “test computer” too

Tom Elliott

@Seydoo Did you remove the client then reinstall, or just “modify/repair”?

Seydoo

@Tom-Elliott
I removed the client then reinstall

There is something strange with one host, lots of MAC are pending :

MFMOA119	98:e7:f4:5a:c8:a2	fe:f3:2d:6a:05:92
MFMOA119	98:e7:f4:5a:c8:a2	0e:7e:b6:29:fd:a4
MFMOA119	98:e7:f4:5a:c8:a2	3a:25:b0:aa:db:85
more lines (around 40) with different host pending MAC

I removed and reinstalled the client on this computer but there is still a lots of pending MAC

Joe Schmitt

@Seydoo uninstalling / installing the client requires additional steps (https://wiki.fogproject.org/wiki/index.php?title=FOG_Client#Reset_encryption_data). As for macs, if they are pending, then they do infact exist on that host (they could be tunnel interfaces, VirtualBox interfaces, or any other kind of virtual adapter),

Also be sure to check the fog.log to ensure that it is still the private key not found error, and not a new one now. Just because the client is still not authenticating doesn’t mean the underlying issue hasn’t changed.

Your original issue Private key not found is strictly a server issue. How did you copy over your installation to the new server? On fresh installs or upgrades this issue is not present, indicating its likely a permission issue from when you migrated the server, or something alike. @Tom-Elliott would be best equipped to tell you which files / permissions to check.

Seydoo

@Joe-Schmitt here are my steps

Old server : RH6.5, FOG 1.3 rc23 @IP 172.27.1.1, around 200 clients

export Image DB

1\ install fresh OS (centos 7) @IP 172.27.1.2
2\ download fog 1.3.5 from sourceforge
3\ install FOG
4\ issues to access WEB GUI ==> uninstall php 5.x then install php7
5\ change fog user password (due to network policies) on server / web GUI / FTP (fog settings) / node admin ==> all pwd are the same (not good but my fog server is in a private VLAN)
6\ Change node storage in web GUI ==> from /images to /meti/images, snapins to /meti/snapins
7\ Create new snapins (I didn’t use the export from the old server because I had problems to deploy some snapins due to hash issues)
8\import images.csv, copy images files from old server to new server
9\ stop the old server
10\ change IP of new server ==> 172.27.1.1 (use wiki to do that)
11\ lots of host pending ==> approve them (strange pending with the MFMOA119 as explain, it’s a basic notebook with 2 virtual NIC, a wifi and an ethernet NIC==> 50 pending MAC ==> approve and then delete host, no more pending since midday)

Test deploying phase :

issue with nfs mount ==> solve with your help yesterday ==> deploy success
Test snapins deployment = issue

my computer is the one used for snapins deployment :
first I used ancient client ==> doesn’t work, remove it and install 0.11.11 = nothing change (privacy key …)

==> copy /opt/fog/snapins/ssl from old to new server as you told me

stil doesn’t work (fog.log is a new one i delete the old one)

My fog server is a VM, I have a snapshot of it before 1st fog install (and before php update)

You have my .fogsetting, do you need more info ?

Tom Elliott

@Seydoo From the sounds of things, you have FOG Client set to be started even though the main image is a sysprepped image?

I say this because:

There’s only a few ways for MAC’s as you’ve described to associate to a single host in the manner you’re referring to.

The mac’s that are registering have a common mac address that’s already “approved” with that particular host. This can happen because of Tunnel MAC’s or Virtual Host Mac addresses that would be the same on ALL hosts when imaging occurs.
The hostname for other hosts are identical at the time the FOG Client goes to check in. If the mac’s are not sharing a common mac, and the hostname is not different for every host, as might happen when imaging occurs, when the host goes to check in it’s passing the hostname option and associating all mac’s with that registered hostname.

As you keep saying the “Private key is not found” can you please repost a new fog.log since you’ve already “Reinstalled the client” and have “copied the original fog server’s /opt/fog/snapins folder entirely”?

Since all of this happened, have you reinstalled the FOG Server using the --recreate-Keys as @Joe-Schmitt suggested?

Seydoo

For the multipe pending MAC, no more since midday (I approve this host and then delete it)

here is a new fog.log, I reinstall fog with --recreate-keys and put all my hosts in a group to reset encryption data, then start fog Service on my computer

26/04/2017 17:24 Main Overriding exception handling
 26/04/2017 17:24 Main Bootstrapping Zazzles
 26/04/2017 17:24 Controller Initialize
 26/04/2017 17:24 Zazzles Creating main thread
 26/04/2017 17:24 Zazzles Service construction complete
 26/04/2017 17:24 Controller Start

 26/04/2017 17:24 Service Starting service
 26/04/2017 17:24 Bus Became bus server
 26/04/2017 17:24 Bus {
  "self": true,
  "channel": "Status",
  "data": "{\r\n  \"action\": \"load\"\r\n}"
}
 26/04/2017 17:24 Bus Emmiting message on channel: Status
 26/04/2017 17:24 Service Invoking early JIT compilation on needed binaries

------------------------------------------------------------------------------
--------------------------------Authentication--------------------------------
------------------------------------------------------------------------------
 26/04/2017 17:24 Client-Info Version: 0.11.11
 26/04/2017 17:24 Client-Info OS:      Windows
 26/04/2017 17:24 Middleware::Authentication Waiting for authentication timeout to pass
 26/04/2017 17:24 Middleware::Communication Download: http://172.27.1.1/fog/management/other/ssl/srvpublic.crt
 26/04/2017 17:24 Data::RSA FOG Server CA cert found
 26/04/2017 17:24 Middleware::Authentication Cert OK
 26/04/2017 17:24 Middleware::Communication POST URL: http://172.27.1.1/fog/management/index.php?sub=requestClientInfo&authorize&newService
 26/04/2017 17:24 Middleware::Response Private key not found

------------------------------------------------------------------------------
--------------------------------Authentication--------------------------------
------------------------------------------------------------------------------
 26/04/2017 17:24 Client-Info Version: 0.11.11
 26/04/2017 17:24 Client-Info OS:      Windows
 26/04/2017 17:24 Middleware::Authentication Waiting for authentication timeout to pass

Tom Elliott

@Seydoo Okay can you run:

sudo chmod -R 755 /opt/fog/snapins
sudo chown -R fog:root /opt/fog/snapins

Then look on the logs?

Seydoo

nothing change

Tom Elliott

@Seydoo Can you get output of:

ls -lharRt /opt/fog/snapins

Seydoo

@Tom-Elliott said in Snapins blocked and remain in "Checked in":

ls -lharRt /opt/fog/snapin

/opt/fog/snapins:
total 12K
drwxr-xr-x 3 fog root 4,0K 25 avril 17:53 .
drwxr-xr-x 3 fog root 4,0K 25 avril 17:53 ssl
drwxr-xr-x 5 fog root 4,0K 26 avril 17:42 ..

/opt/fog/snapins/ssl:
total 20K
drwxr-xr-x 3 fog root 4,0K 25 avril 17:53 ..
drwxr-xr-x 2 fog root 4,0K 25 avril 17:53 CA
drwxr-xr-x 3 fog root 4,0K 25 avril 17:53 .
-rwxr-xr-x 1 fog root 3,2K 26 avril 17:22 .srvprivate.key
-rwxr-xr-x 1 fog root 1,6K 26 avril 17:22 fog.csr

/opt/fog/snapins/ssl/CA:
total 20K
-rwxr-xr-x 1 fog root 1,8K 23 sept.  2016 .fogCA.pem
-rwxr-xr-x 1 fog root 3,2K 23 sept.  2016 .fogCA.key
drwxr-xr-x 2 fog root 4,0K 25 avril 17:53 .
drwxr-xr-x 3 fog root 4,0K 25 avril 17:53 ..
-rwxr-xr-x 1 fog root   17 26 avril 17:22 .srl

.srvprivate.key and fog.scr change with --recreate-keys ? because the date is not the same (today instead of 09/2016)

Tom Elliott

@Seydoo What’s defined for the SSL path in the storage node?

Seydoo

@Tom-Elliott
…
O.M.G !!!
In french I’ll say “I’m a milstone”
I completely forgot check this value (which I changed when configure my storage node)
I’m so sorry for the lost time…
Of course the path wasn’t correct, i change it with /opt/fog/snapins/ssl

Tom Elliott

@Seydoo so updating the SSL path in the storage node made things work?

Snapins blocked and remain in "Checked in"

Server

Client

Description

148

12.0k

17.3k

155.2k