Posts made by tian

tian

Some change between git5221 and git5283 seemed to have solved the hostMAC table problem.

I cant see the increase anymore in git5283. Since the hostname changer service seemed to be related to it I tested the AD join and it still seems to work fine.

I’ll test some more, but it looks good now.

tian

There is a bug in git5135 (and maybe earlier) I found during testing something else before:

A Service in the host menu is displayed as activated
deactivating one or more service by unchecking it
clicking update
->“Host Updated!” appears on top of the page but the service is activated again.

I tried with all four Services globally enabled - none of them could be deactivated for a host.
Activating a Service did work before. I tested it with Firefox and Internet Explorer 11.

tian

@Tom-Elliott somewhere between git5122 and git 5135 seemed to have an effect on the hostMAC table:
With git5135 the hostMAC changes the following:

Windows login screen/no user logged in on host: avg. ~1.5 changes per minute/client
user logged in to Windows: avg. ~2.5 changes per minute/client

With each change/rewrite of the hostMAC entry of the client computer turned on the hmID and Auto_increment rises even more fast than before now.

I found out that the Hostname Changer service could be involved - because for one host the service wasn’t activated. The client computer only changed the hmID/Auto_Increment only once when windows started up - not anymore when just running.
After activating the Hostname Changer service for this host the hmID for this host was increased immediately like described above.

tian

@Tom-Elliott Thanks - git5135 fixed the snapin log:

“active snapins” are added/removed correctly -> OK
snapins are running ok on the client -> OK
the snapin log displays the database content correctly again (it added the new snapin tasks additionally to the ones from yesterday)** -> OK**
the snapin status still is reported as “4” in snapinJobs and snapinTasks -> OK?
no “sjHostID 0” entries in snapinJobs are created from hosts only started up/running -> OK

tian

After updating from git5122 to git5135 the following screen appears instead of the web page where it tells you where to log in to fog:
2015-10-30 fog 5122 to 5135.png

The two values mentioned there are not in the table globalSettings anymore. Logging in to fog works without problems. The two settings are not there anymore in fog settings. Can the error message be ignored?

tian

@Tom-Elliott said:

@tian was the snapin completed today?

Yes - the snapins have been completed on the host. I could verify the changes have been made there.
The snapins with the ID 1 and 11 are simple batch files with some simple commands - the snapin with ID11 additionally contains wait commands for 90 seconds in total.

tian

@Tom-Elliott

on the web interface (Taskmanagement -> OK
– the active snapin task was there
– after running the snapin the task vaished
on the web interface (snapin log -> not OK
– i can select the date range of today
– no log of the just run snapin was found (“no results found”)
– the csv export is empty (just the headers)
on the client (OK
– the client icon messages that the snapin(s) is/are running or finish appear when a user is logged in
– the client is able to see the snapin and runs it - return code of the batch file is 0 (information from the fog.log on the client)
in the database ((not) OK??
– snapinJobs:

+------+----------+-----------+---------------------+
| sjID | sjHostID | sjStateID | sjCreateTime        |
+------+----------+-----------+---------------------+
|    1 |       21 |         4 | 2015-10-29 12:40:15 |
|    2 |       21 |         4 | 2015-10-29 12:59:37 |
+------+----------+-----------+---------------------+

– snapinTasks:

+------+---------+---------+---------------------+---------------------+------------+--------------+-----------------+
| stID | stJobID | stState | stCheckinDate       | stCompleteDate      | stSnapinID | stReturnCode | stReturnDetails |
+------+---------+---------+---------------------+---------------------+------------+--------------+-----------------+
|    1 |       1 |       4 | 2015-10-29 12:40:30 | 2015-10-29 12:40:31 |          1 |            0 |                 |
|    2 |       2 |       4 | 2015-10-29 13:00:00 | 2015-10-29 13:00:01 |          1 |            0 |                 |
|    3 |       2 |       4 | 2015-10-29 13:00:01 | 2015-10-29 13:01:36 |         11 |            0 |                 |
+------+---------+---------+---------------------+---------------------+------------+--------------+-----------------+

I started the “All Snapins” at advanced actions two times for this host. the first time with one snapin, the second time with 2 snapins on the host.
Running a single snapin with the single snapin advanced action behaves the same like above.

tian

I did a quick test with git5122 - (two computers turned on, Snapin service activated, no snapin task) - and it seems to be fixed:

no new “sjHostID 0” entries in snapinJobs are created anymore - also not after truncate snapinJobs

I try to test some more with snapins and report if anything doesn’t work.

tian

I just tested with git5074 and the entries with sjHostID 0 still are created, when a host computer is turned on.

On the host computer the client checks for snapins every minute without any error:

------------------------------------------------------------------------------
---------------------------------SnapinClient---------------------------------
------------------------------------------------------------------------------
 26.10.2015 10:22 Client-Info Version: 0.9.5
 26.10.2015 10:22 SnapinClient Running...
 26.10.2015 10:22 Middleware::Communication URL: http://SERVER-IP/fog/service/servicemodule-active.php?moduleid=snapinclient&mac=ONBOARD-MAC|VM-MAC|&newService=1
 26.10.2015 10:22 Middleware::Communication Response: Success
 26.10.2015 10:22 Middleware::Communication URL: http://SERVER-IP/fog/service/snapins.checkin.php?mac=ONBOARD-MAC|VM-MAC|&newService=1
 26.10.2015 10:22 Middleware::Communication Response: No snapins
------------------------------------------------------------------------------

More things I tested:

truncated the Table snapinTasks -> “sjHostID 0” entries still created
– (I still think about why there is/was status code “4” - I just remember 1 for running and 2 for finished)
disabled the Snapins service in the Service Configuration of one host -> for this host no “sjHostID 0” entries are created anymore
– enabling the Snapin service for this hosts again -> “sjHostID 0” entries are created every minute again for this host

Which table content/error log i can provide to help to find out why that is caused?

tian

In the table snapinJobs entries with (sj)HostID appear when a host computer (with client 0.95) is turned on:

+------+----------+-----------+---------------------+
| sjID | sjHostID | sjStateID | sjCreateTime        |
+------+----------+-----------+---------------------+
|    1 |        0 |         5 | 2015-10-22 11:32:44 |
|    2 |        0 |         5 | 2015-10-22 11:37:01 |
|    3 |        0 |         5 | 2015-10-22 11:38:04 |
|    4 |        0 |         5 | 2015-10-22 11:39:06 |
+------+----------+-----------+---------------------+

Between 11:32 and 11:37 the computer was restarted. Before the restart I truncated the table snapinJobs.

It isn’t triggered only by a special host computer turned on. I tested it with more computers
The table “hosts” begins with hostID 1.
On the web interface there are no open/waiting snapin tasks.
The table “snapinTasks” contains some already finished tasks with “stState” -> “4”.
The table “snapinAssoc” contains just entries with saHostID beginning from “1”.

Is there any more information needed to solve this?

tian

@Tom-Elliott Thanks for the fix. It works now (git5040).

For the Database changes - they seem to be OK too:

snapinAssoc: removed snapin associations are deleted from table, Auto_Increment not increased
hostMAC: the table entries for the hosts the snapin has been removed have been rewritten -> new hmIDs, Auto_Increment increased by the number of hosts in the group

tian

When trying to remove a snapin from the hosts of a group by the group menu the message “Group information updated!” appears on top of the page, but no snapins have been removed from the hosts in the group.

Group-adding snapin works as does single remove of snapins for hosts.

tian

@Tom-Elliott Thanks a lot for the change.

I now had time to test again (with git5028 - I just verified the problematic tables groupMembers, hostMAC, moduleStatusByHost and SnapinAssoc) and can give some feedback now:

removing/adding a snapin from a group
– only the hostMAC-Table increases the “Auto_increment” value by the number of hosts the snapin is removed added (in my case 19 in both cases)
removing/adding computers from/to a group
– removing hosts did not increase the "Auto_increment"value of these tables, adding the same 2 hosts to the same group again increases the “Auto_increment” value of hostMAC by 2. The “Auto_increment” value of the 3 other tables are not increased

This behavior seems to be fine

But one thing is still there:

no changes by the web interface are done/no user is logged in to the web interface
host computers with client 0.95 are started to windows 7 - a user logged in or not doesn’t make a difference
– The result of this setup is that the hostMAC entry of the computers that are turned on is changed/refreshed around every 2 minutes for every computer
With the “change” (where -however- nothing seems to be changed at all, excepet the hmID) the “Auto_increment” value of hostMAC also increases by 1 for every computer every 2 minutes. (so 2 computers update the hostMAC database every minute by one entry in average)

Can this be reduced too, or is this necessary for other fog processes?
If it is important - the Computers only have 1 NIC and a virtual NIC (I blocked these from being pending MAC addresses by using FOG_QUICKREG_PENDING_MAC_FILTER)

tian

@Tom-Elliott Thanks for your patience and this great piece of software. I understood that by changing, deleting , … an object in fog causes deleting and re-entering the database entries (a still existing fog 0.32 server also has relatively high IDs there - but these are still much lower -> a few thousand compared to a few 100k on the test-server for current svn).

I just wondered about the loop of deleting and re-entering, even with the computers turned off the whole time (so no AD join or other fog-client actions could be there - and me is the only one on this fog server for testing)

But the good new is that I found out what was causing the loop. The mysql services was running several processes (mysqld_safe, mysqld with some parameters, mysqld without parameters).
Running dpkg-reconfigure for the mysql server solved this and now just one mysqld process without parameters is running.

It seems to run stable now - for testing i:

removed a snapin from a group of 19 -> The IDs/auto_increment of groupMembers get up by 19, the IDs of snapinAssoc get up by 18
added the same snapin again -> The IDs/auto_increment of groupMembers get up by 19, the IDs of snapinAssoc get up by 37
(a second snapin is associated to only 18 of the 19 group members, so the 18 and 37 from the snapinAssoc should be correct)

The reason why I didn’t check the mysqld service directly was that everything seemed to run correctly - imaging was working, the web interface was ok, the database-update when updating the svn version was returning a OK message, … . Even when installing a new svn over a existing svn version there was no errors on the mysql actions displayed.

Think I really have to check every time after each Ubuntu update/restart and fog svn update, if the myqld service is running just one process and check the database more often. (mysqld and other services are already delayed at startup)

tian

@Tom-Elliott said:

This is normal and expected behavior.

While these 21 systems that weren’t doing anything should not have had anything updated, the systems that were still in the system and pinging FOG would. This is because:

The system initially destroy’s ALL associations of a specified Object (Host with Snapins, Printers, MAC’s, etc…). This way the updates you perform that remove an element, that element is removed, and new entries (of the items you require for that object) are created even though they existed before. This “autoincrements” the fields continuously.

Thanks Tom for the explanation.

Tomorrow I’ll try if imaging and everything else is working with the re-imported hosts.

Just to clarify - I didn’t update the hosts anymore after associating the group, snapins, images, services and ad join information to them hours ago. That it is still increasing every few minutes is normal behavior?

If that is normal there is something that bothers me:
Currently there are just 21 hosts and 1-2 snapins connected to each host for testing. If more hosts and snapins will be added and associated to the hosts, the increase of the values will be faster. At the moment the auto_increment values / IDs range from ~1200 to 4800 again.

What happens if auto_increment hits its limit - will it begin from 1 again?
And if it begins from 1 again - is it for sure that all the IDs will be incremented at any time without forgetting some that will remain at some lower value then? (-> double IDs?)

tian

There seems to be bug that increases the IDs of the following tables constantly:
groupMembers (gmID)
hostMAC (hmID)
moduleStatusByHost (msID)
snapinAssoc (saID)

The IDs of the table elements are increasing - the number of rows keep being the same (the previous IDs are not there anymore).

What I did before (because of IDs in these 4 tables had a higher value than the corresponding Auto_increment values …/ the IDs and auto_increment values have been from around 196k to 681k …):

I completely cleared (by trucnate) the database from hosts, group and snapin association tables. -> this also reset all the the auto_increment to 1
– (groupMembers, hostMAC, hosts, inventory, ipxeTable, moduleStatusByHost, SnapinAssociation, snapinGroupAssoc, snapinJobs, scheduledTasks, taskLog, tasks, snapinTaks, multicastSessionAssoc, imagingLog) - I just kept the tables groups, images and snapins (where the IDs/auto_increment and data still are fine)
after that I re-imported the hosts by a csv file (21 hosts, just mac and name in csv)
associated the hosts to the groups
associated the image, services and AD join data
(the hosts have been turned off the whole day)

now (around 1h after re-adding the 21 hosts) the values for auto_increment and IDs reach from 236 to 1007 again - without changing anything on the web interface and even being logged out.

During cleaning the database I was on GIT4798 and updated to GIT4804 with the same behavior. I’m on Ubuntu 14.04 with mysql 5.5.44.
Fog seems to work fine even with this bug - Imaging, Snapins, AD join work with GIT4798 yesterday.

Is there anything more I can try or provide?

tian

Today I updated to SVN3973/GIT4486 and re-entered the (same) AD join data and reset the host encryption keys and suddenly it works again:

...
 10.09.2015 14:19 HostnameChanger Checking Hostname
 10.09.2015 14:19 HostnameChanger Hostname is correct
 10.09.2015 14:19 HostnameChanger Registering host with active directory
 10.09.2015 14:19 HostnameChanger Success, code =  0
 10.09.2015 14:19 Power Creating shutdown request
 10.09.2015 14:19 Power Parameters: /r /c "Host joined to Active Directory, restart required" /t 0
 10.09.2015 14:19 HostnameChanger Activing host with product key

The log entries after this still contain “Middleware::Communication Response: Invalid host certificate”, but additionally also “HostnameChanger The machine is already joined to the domain, code = 2691” is there instead of “ERROR: Required Domain Joining information is missing”.

going to:
http://IP.OF.FOG.HERE/fog/service/hostname.php?mac=MACOFHOSTTOJOINAD
and
http://IP.OF.FOG.HERE/fog/service/hostname.php?mac=MACOFHOSTTOJOINAD&newService
resulted in the same outputs as when it was not working.

I really don’t know why it is working now - on the client computer nothing was changed since yesterday.

tian

@Tom-Elliott said:

@tian said:

“http://IP.OF.FOG.HERE/fog/service/hostname.php?mac=MACOFHOSTTOJOINAD”

Of course blocking out sensitive information, what is the output if you goto:
http://IP.OF.FOG.HERE/fog/service/hostname.php?mac=MACOFHOSTTOJOINAD&newService

Is the password correct? This may or may not work properly, as it should return either IHC (Invalid Host Certificate) or #enkey=RandomGarble

Thanks for your help.

Going to the page you mention I get “#!ihc” as a result.

The domain user to join the domain is working correctly when joining the domain manually. The password/user is the same as I entered in fog (username without domain).

(The Hostname Changer services is activated and the computer’s name gets renamed correctly based on the name in fog. The computer I get the Image from is removed from the domain and restarted before uploading.)

Maybe tomorrow or on Friday (or next week …) I will try the newest SVN/GIT and check if anything is different to the currently installed SVN/GIT.

tian

some additional information:

After:

filling the “Active Directory Defaults” with fake AD and User Data
Save
filling in the “Active Directory Defaults” values from before (including the FOG_AD_DEFAULT_PASSWORD to re-encrypt)
save
resetting the host encryption data
the log entry changed to the following (for once only - the HostnameChanger log entries after this look like the one I posted before)

------------------------------------------------------------------------------
--------------------------------HostnameChanger-------------------------------
------------------------------------------------------------------------------
 04.09.2015 12:46 Client-Info Version: 0.9.4
 04.09.2015 12:46 HostnameChanger Running...
 04.09.2015 12:46 Middleware::Communication URL: http://IPofFOGSERVER/fog/service/servicemodule-active.php?moduleid=hostnamechanger&mac=MACfromNIC|MACfromVirtualBox|&newService=1
 04.09.2015 12:46 Middleware::Communication Response: Success
 04.09.2015 12:46 Middleware::Communication URL: http://IPofFOGSERVER/fog/service/hostname.php?moduleid=hostnamechanger&mac=MACfromNIC|MACfromVirtualBox|&newService=1
 04.09.2015 12:46 Middleware::Communication Response: Success
 04.09.2015 12:46 HostnameChanger Checking Hostname
 04.09.2015 12:46 HostnameChanger Hostname is correct
 04.09.2015 12:46 HostnameChanger Registering host with active directory
 04.09.2015 12:46 HostnameChanger ERROR: Required Domain Joining information is missing
 04.09.2015 12:46 HostnameChanger Activing host with product key
------------------------------------------------------------------------------```

tian

The same problem seems to happen with SVN3959/GIT4557.

Here is the HostnameChanger part of the log on the client:

------------------------------------------------------------------------------
--------------------------------HostnameChanger-------------------------------
------------------------------------------------------------------------------
 03.09.2015 13:32 Client-Info Version: 0.9.4
 03.09.2015 13:32 HostnameChanger Running...
 03.09.2015 13:32 Middleware::Communication URL: http://IPofFOGSERVER/fog/service/servicemodule-active.php?moduleid=hostnamechanger&mac=MACfromNIC|MACfromVirtualBox|&newService=1
 03.09.2015 13:32 Middleware::Communication Response: Success
 03.09.2015 13:32 Middleware::Communication URL: http://IPofFOGSERVER/fog/service/hostname.php?moduleid=hostnamechanger&mac=MACfromNIC|MACfromVirtualBox|&newService=1
 03.09.2015 13:32 Middleware::Communication Response: Invalid host certificate
 03.09.2015 13:32 Middleware::Communication URL: http://IPofFOGSERVER/fog/management/other/ssl/srvpublic.crt
 03.09.2015 13:32 Data::RSA CA cert found
 03.09.2015 13:32 Middleware::Authentication Cert OK
 03.09.2015 13:32 Middleware::Communication POST URL: http://IPofFOGSERVER/fog/management/index.php?sub=authorize
 03.09.2015 13:32 Middleware::Communication Response: Success
 03.09.2015 13:32 Middleware::Authentication Authenticated
 03.09.2015 13:32 Middleware::Communication URL: http://IPofFOGSERVER/fog/service/hostname.php?moduleid=hostnamechanger&mac=MACfromNIC|MACfromVirtualBox|&newService=1&newService=1
 03.09.2015 13:32 Middleware::Communication Response: Success
 03.09.2015 13:32 HostnameChanger Checking Hostname
 03.09.2015 13:32 HostnameChanger Hostname is correct
 03.09.2015 13:32 HostnameChanger Registering host with active directory
 03.09.2015 13:32 HostnameChanger ERROR: Required Domain Joining information is missing
 03.09.2015 13:32 HostnameChanger Activing host with product key
------------------------------------------------------------------------------

The srvpublic.crt exists and was last changed with the update to SVN3959

Going to “http://IP.OF.FOG.HERE/fog/service/hostname.php?mac=MACOFHOSTTOJOINAD” results in:

#!ok=OURCOMPUTERNAME #AD=1 #ADDom=OURDOMAIN #ADOU= #ADUser=OURDOMAIN\OURUSER #ADPass=

The last Version of FOG I remember the Domain Join worked was SVN3901/GIT4457. Resetting the Encryption Data doesn’t help.

The Legacy Domain Password is empty since we just use the new Client with the new FOG Server. The Image (Windows 7 Prof.) contains the FOG-Client 0.94.