Migrate Hosts from 1.2 to dev-trunk

jayphizzle

Hi!

i’m just setting up a new fog server with the actual dev version.
I exported the hosts from the old server and imported the csv file at the new server.

The problem is that only the hostname and the primary mac was imported.
So i’m missing the host description and the host product key.

I tried to copy the product key via phpmyadmin from the old db to the new db.
But they don’t show up in the webgui.
Then i saved a product key in the webgui and saw that they seem to be not encoded by base64 anymore.
It that correct ?

So is there a way to encode the product keys to the new “format”?
(direct from base64 or via the plain text)

TD;LR
How are the product keys in the dev versions encoded?

Sebastian Roth

The product keys are AES encrypted in the trunk/dev version of FOG. So unfortunately you cannot simply import this information using phpmyadmin. The import function is not doing it either as I can see.

If you are keen I guess you could come up with some shell script magic that will convert from base64 to AES encrypted string. Check out fogbase.class.php (line 366-391)

I have to admit that I joined the FOG team while trunk development was at full speed and didn’t have the time to look into all the details of the web development. So I am not sure if Tom already has a solution to this. If not we might work out a nice script to do the conversation… or improve the import function to detect non-AES encoded strings and do the encoding on the fly.

Sebastian Roth

I had a closer look at the code and I think we can add a fairly simple check to see if the imported product key data is already encrypted or not (then encrypt it before writing to database). What do you think about this @Tom-Elliott? I guess we will see more and more people wanting to migrate their 1.2.0 server to a new machine…

diff --git a/packages/web/lib/fog/fogpage.class.php b/packages/web/lib/fog/fogpage.class.php
index 7ab7654..2f0cfa8 100644
--- a/packages/web/lib/fog/fogpage.class.php
+++ b/packages/web/lib/fog/fogpage.class.php
@@ -991,6 +991,13 @@ abstract class FOGPage extends FOGBase {
                     if ($Item->getManager()->exists($data[$iterator])) throw new Exception(sprintf('%s %s: %s',$this->childClass,_('already exists with this name'),$data[$iterator]));
                     foreach (array_keys((array)$this->databaseFields) AS $i => $field) {
                         if ($Item instanceof Host) $i++;
+                        if (isset($field) && $field === 'productKey') {
+                            $test_encryption = $this->aesdecrypt($data[$i]);
+                            if ($test_base64 = base64_decode($data[$i])) {
+                                $data[$i] = $this->aesencrypt($test_base64);
+                            }
+                            elseif (empty($test_encryption) || !mb_detect_encoding($test_encryption,'UTF-8',true)) {
+                                // importated productKey data seams to be not encrypted, so we do it
+                                $data[$i] = $this->aesencrypt($data[$i]);
+                            }
+                        }
                         $Item->set($field,$data[$i],($field == 'password'));
                     }
                     if ($Item instanceof Host) {
diff --git a/packages/web/lib/fog/host.class.php b/packages/web/lib/fog/host.class.php
index decef5e..2c7841f 100644
--- a/packages/web/lib/fog/host.class.php
+++ b/packages/web/lib/fog/host.class.php
@@ -19,8 +19,8 @@ class Host extends FOGController {
         'ADPassLegacy' => 'hostADPassLegacy',
         'productKey' => 'hostProductKey',
         'printerLevel' => 'hostPrinterLevel',
-        'kernel' => 'hostKernel',
         'kernelArgs' => 'hostKernelArgs',
+        'kernel' => 'hostKernel',
         'kernelDevice' => 'hostDevice',
         'pending' => 'hostPending',
         'pub_key' => 'hostPubKey',

The change in host.class.php is fixing a minor bug that I found while testing this. hostKernel and hostKernelArgs are in reverse order in the database. Usually the wrong order in that array does not cause any trouble I suppose. But when you create your own CSV file according to the DB structure and try to import you’ll end up with kernel and kernelArgs being “swapped”.

@jayphizzle The exported CSV from FOG 1.2.0 was kind of minimalistic (only "mac", "name", "ip", "description", "imageID"). So you need to generate a proper CSV from your 1.2.0 database via mysql select statement. Try this via phpmyadmin or mysql cli:

SELECT hostMAC.hmMAC,hosts.hostName,hosts.hostDesc,hosts.hostIP,hosts.hostImage,hosts.hostBuilding,hosts.hostCreateDate,hosts.hostLastDeploy,hosts.hostCreateBy,hosts.hostUseAD,hosts.hostADDomain,hosts.hostADOU,hosts.hostADUser,"",hosts.hostADPass,hosts.hostProductKey,hosts.hostPrinterLevel,hosts.hostKernelArgs,hosts.hostKernel INTO OUTFILE '/tmp/result.csv' FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' FROM hostMAC,hosts WHERE hostMAC.hmHostID=hosts.hostID;

Examine the SQL statement carefully and you’ll see that it dumps the results into the file /tmp/results.csv on your server.

#wiki Marking this to go to the wiki at some point because I am sure we’ll see people moving from 1.2.0 as soon as a new version is officially released! Keeping my fingers crossed that most people will migrate by installing “on top” where no export/import of the database is needed!!

Sebastian Roth

@jayphizzle Have you tried upgrading your trunk server and importing the mysql-generated CSV yet? I tested a lot but I am still wondering if this is really working for other users as well! Please let us know.

Tom Elliott

@Sebastian-Roth I only just finally added your suggestions. Literally like 32 seconds ago. Sorry I saw it and just missed it.

jayphizzle

@Sebastian-Roth said:

@jayphizzle Have you tried upgrading your trunk server and importing the mysql-generated CSV yet? I tested a lot but I am still wondering if this is really working for other users as well! Please let us know.

@Sebastian-Roth Sorry i did not give feedback to your suggestion. I tried it but the product key field in the webgui does not show any entry.

@Tom-Elliott said:

@Sebastian-Roth I only just finally added your suggestions. Literally like 32 seconds ago. Sorry I saw it and just missed it.

Damn. I entered the product keys for most of our clients manually yesterday.
I have to be more patient next time

I tested the new trunk version just a minute ago.

As mentioned i imported the base64 coded productkey from my 1.2 installation into the db of the dev-version.
I checked the productkey of a host i did not updated manually - field is still empty.
I clicked on “update” in the host view - field is still empty and also the db field is now empty.
I copied the base64 coded product key from the db an entered it in the field in the host view - there is a encoding of the entered key, but it is not correct. (Q1RRW-XXXXX-XXXXX-XXXXX-XXXXX instead of CTQY2-XXXXX-XXXXX-XXXXX-XXXXX )

Sebastian Roth

@jayphizzle said:

Damn. I entered the product keys for most of our clients manually yesterday.
I have to be more patient next time

Is a response time of 24 hours for a complete solution not fast enough?!? Just kidding!

The proposed fix is only able to jump in and recognize/modify the values being imported on the fly. I don’t think we want to have some foo magic that tries to make assumptions on the product key filed every time it’s loaded/saved from/to the database. So your tests (import base64 via db dump and update the value via web gui) just won’t work - as you already saw!

If I’d be you, I’d go for this: Do a complete mysql-CSV-dump from your 1.2.0 DB as described earlier. Make a copy of that full CSV file and delete most lines to only keep a few (maybe 3-5) for testing. Choose whichever you like (maybe hosts that you haven’t modified the product key value by hand yet so you won’t ruin those precious entries you spend hours on I suppose). Then delete exactly those 3-5 hosts from your FOG trunk DB using the web gui and import the CSV. Check if everything is fine - especially AD passwords as I don’t think we have that correctly yet - and let us know!

jayphizzle

@Sebastian-Roth said:

The proposed fix is only able to jump in and recognize/modify the values being imported on the fly.

Ah ok only during the import.
Why i did not tried this

I tried it with a few hosts as you recommended - the productkey was successfull decoded from base64 and saved in AES.
(i did not test ad passwords cause the last hosts where standalone devices)

Thanks to @Sebastian-Roth and @Tom-Elliott for the quick (bug)fix!

Sebastian Roth

Anyone interested to test this who has settings like AD passwords and such?

Sebastian Roth

@Testers Anyone still running FOG 1.2.0 anywhere? Would be great if you could run through the export/import process described below to see if it all works… I guess we still have problems with the AD passwords.

Tom Elliott

@Sebastian-Roth Should we adjust the export method you described to place the ADPasswordLegacy into the proper location on the CSV?

Sebastian Roth

Chatted with Tom about this. I didn’t have in mind that ADPass in FOG 1.2.0 was encrypted using FOGcrypt (different encryption than we use in FOG trunk now!) and that value stored in the DB. This value needs to go to the field ADPassLegacy in FOG trunk without any conversation while ADPass in trunk needs to stay empty as we cannot decrypt the old value ( @Tom-Elliott sure we can’t do this?).

I changed my earlier posting to reflect this. Still would be great if anyone could test and let us know!

Wayne Workman

For future readers: https://wiki.fogproject.org/wiki/index.php?title=Migrate_FOG

Migrate Hosts from 1.2 to dev-trunk

195

11.7k

17.2k

154.7k