Migrate PM to VM 1.2.0



  • I am having trouble migrating to a VM.

    I backed up the database in Fog Configuration --> Configuration Save.

    And backed up the \images folder.

    I made the VM, installed the same version of fog on the VM, imported the database, and moved the images folder over.

    I also did what i read on this page https://community.spiceworks.com/how_to/2631-moving-a-fog-server-to-new-hardware

    Copy the contents of the images folder which is /images and place them in the /images folder on the new server.

    (This is assuming that you are using the default storage and have not set up any additional)

    After that you will need to change the permissions:

    sudo chown -R fog /images
    sudo chmod -R 755 /images
    sudo chmod -R 777 /images/dev

    note: When I did my first run on the new server I got a failure to mount nfs volume error the fix is

    touch /images/dev/.mntcheck
    touch /images/.mntcheck

    I did all of that and my images are still not showing up. And I dont even think the database export/import took either because none of my settings in the Fog Confiuration --> Fog Settings page showed up.

    I either don’t understand the capabilities of the database export/import, or it didnt work.

    I thought from the page here https://forums.fogproject.org/topic/2714/info-on-fog-server-disaster-recovery/3
    If you make backups of your Images folder (just copy paste, or compress a copy and then move it) and your database, then you have everything you really need.

    That this would be all I need, and the database export/import would take care of everything except the images folder.

    Am I incorrect assuming this?

    Any help is greatly appreciated!


  • Moderator

    I’ve updated this article to show how to move all images instead of just one. I’ve also linked it to the FOG Client article, and vice versa.

    https://wiki.fogproject.org/wiki/index.php?title=Migrate_images_manually


  • Moderator

    @Wayne-Workman Updated steps are in the wiki in the below link for migrating SSL certificates. The directory locations of these files have changed in the last few months, so please follow the wiki’s examples.

    https://wiki.fogproject.org/wiki/index.php?title=FOG_Client#Maintain_Control_Of_Hosts_When_Building_New_Server


  • Moderator

    @MRCUR The method I described is a way to leave behind settings that are unique to the old server.

    The “Configuration Save” option inside the FOG Configuration area does not export the CA. The CA is rather important to protect. It’s what’s used to generate ssl certificates, you don’t want it so easily downloaded. Same goes for the certs.

    The configuration save has it’s uses. I use it for backup purposes, I would not use it to migrate a fog server’s settings. Say if I’m playing around with the database manually via CLI, it’s a good idea to do a config save first. That way, I can go back easily without having to do something as drastic as restoring a snapshot.

    I always thought that locations and storage nodes were really easy to set up…

    also - I updated that below post a little.


  • Testers

    @Wayne-Workman Thanks Wayne. That’s just the info I was looking for.

    At this point, is the built-in config export/import not really needed anymore, or has it not been updated to handle the new CA/SSL pieces? I’m fine with your method since I’m not looking to migrate “bad” settings/config info, but I do use the location plugin and I’m not sure your method would move that data over.


  • Moderator

    @MRCUR Thanks for bumping this thread.

    Tagging this for the wiki so I don’t forget about it again.

    But basically, for 1.3.0 (currently in development):

    inside of /opt/fog/snapins/CA you’ll find your CA. This is the important piece that you want to save.

    (You might want to copy /opt/fog/snapins/ssl too)

    You’d use the web interface to export just your hosts and images. These can both be done from the host management page and the image management page.

    Next you want to save your images. There are a large number of ways to do this - but NFS is the most simple as it doesn’t require credentials and the existing FOG server should already have the images exported via NFS.

    On the new server, the commands to mount and copy would be:

    cd /
    mkdir /images
    mkdir tempMount
    mount x.x.x.x:/images /tempMount
    cd /tempMount
    cp -R *.* /images
    umount /tempMount
    

    However this can also be done recursively with FTP, and can be done with Samba as well. I’ve built a project that installs and configures Samba specifically for a fog server. It’s on sourceforge called “Samba for Fog”.

    When you install fog on the new server, just go ahead and install it initially like normal. After it’s installed, you can swap out the CA for the old one. Then you’d run the installer again - this will re-create new certificates for you, based on that CA.

    Then you’d import your images FIRST in Image management, and then import your hosts second in host management.

    Importing images is a little funny, the ID numbers get re-set to begin at zero. For instance I have several images, all begin in the 30s and up. Re-importing those would reset all the IDs. I’m not sure if it matters at all, honestly. Just something I’ve noticed.

    Of course, if your FOG IP changes, then you’ll need to update Option 066 / Next-Server on whatever is doing DHCP. If FOG is doing DHCP, it will configure itself for this. Also, if the IP changes, and you’ve installed the FOG Client using the IP, then all your FOG Clients will stop working (and will need re-installed). If you’ve installed the FOG Client using a FQDN, then all you need to do is update the DNS “A” record for that FQDN to point to the new IP, and make sure your new fog server has the same name as the old one.

    The most important bit of advice I can give is don’t blow away your old server. Leave it alone in case disaster strikes. Only decommission it when you’re sure you’re solid.


  • Testers

    @Wayne-Workman Have you gotten anywhere on a Wiki article on this topic? I found this thread while looking for the best and most up to date info on doing a FOG server migration. I’ve been looking for the best option to migrate from an existing Ubuntu VM running trunk to a new Ubuntu VM running trunk but don’t want to break anything given FOG runs its own CA now.


  • Moderator

    @george1421 said:

    The one thing you have to be aware of when you migrate, you have to have the same version of FOG on both sides of the migration. For example if you try to migrate from 1.2.0 stable to 1.2.0 stable, you are pretty assured that the data will migrate without issue. But if you try to migrate from 0.32 to 1.2.0 the migration will fail without question. The same holds true if you try to migrate from 1.2.0 stable to 1.2.0 trunk 5676. (understand this is only used to prove a point). The database has changed quite a bit from 1.2.0 to any one of the trunk builds.

    I know. I intend to cover this.

    In the case of the OP, I think we need to look into why the database is not exporting and importing as expected. Something is not right here. At the very least we can use the process outlined by Arrowhead-IT to migrate the data. But I have a strong belief that this should be done from within the GUI to keep people out of the underlying OS whenever possible.

    I’m talking about a total migration. Leaving one OS and going to another, and taking your data, certs, and settings with you. You can’t avoid interacting with the OS for this, if you have many hosts and rely on FOG to do many things you simply cannot abandon your certs and CA now.


  • Moderator

    @Tom-Elliott You know I’ve been using the trunk version for so long I forgot all about the default file size limitation/setting in php. When I did the import / export test on the 1.2.0 instance I spun up so I could help with 1.2.0 questions. I downloaded, deleted a host reference and uploaded the saved copy and the host came back. But this system only has 2 images and 5 hosts. So the exported database wasn’t very big at all. If the OP has many systems or images I could see how it might reject the import as you stated. Good catch, I think you are spot on.


  • Senior Developer

    @george1421 and all I’m just guessing as to the problem as error logs would go a much longer way. First my guess is the issue is not the import directly, but rather the filesize upload limit set by php. In trunk builds I set the values dealing the post max filesize and Max upload size to a fairly decent 100meg limit. However, on 1.2 and earlier this value was left untouched. The post max filesize is defaulted to 8meg and the upload filesize is defaulted to 2meg. My guess is the upload failed because the size of the sql file is larger than the 2meg limit. The post max limits the number of files ( so 4 2meg files) allowed to upload and their combined size.


  • Moderator

    @Wayne-Workman said:

    This thread has given me the idea to create a migration article. We already have an article for changing the IP address of a FOG Server (in the works). In my opinion, that should just be a sub-section. There should be another section in the same article that talks about moving images and other data over to another server.

    The one thing you have to be aware of when you migrate, you have to have the same version of FOG on both sides of the migration. For example if you try to migrate from 1.2.0 stable to 1.2.0 stable, you are pretty assured that the data will migrate without issue. But if you try to migrate from 0.32 to 1.2.0 the migration will fail without question. The same holds true if you try to migrate from 1.2.0 stable to 1.2.0 trunk 5676. (understand this is only used to prove a point). The database has changed quite a bit from 1.2.0 to any one of the trunk builds.

    In the case of the OP, I think we need to look into why the database is not exporting and importing as expected. Something is not right here. At the very least we can use the process outlined by Arrowhead-IT to migrate the data. But I have a strong belief that this should be done from within the GUI to keep people out of the underlying OS whenever possible.


  • Moderator

    @JDnoble18 said:

    I am having trouble migrating to a VM.

    This thread has given me the idea to create a migration article. We already have an article for changing the IP address of a FOG Server (in the works). In my opinion, that should just be a sub-section. There should be another section in the same article that talks about moving images and other data over to another server.

    There are a few ways it can be done - you could do a FULL DB backup - but any problems you have will be migrated too. Or you could do just images,groups,and hosts (in that exact order).

    Migrating the actual image data is quite trivial. There are three ways to do that. One is NFS, two is lftp, three is Samba. I’ve written a Samba installation script configured specifically for installing onto a pre-existing FOG Server, it can be found here: https://sourceforge.net/p/samba-for-fog/svn/HEAD/tree/

    Tagging this idea for addition to the #wiki in the future.


  • Moderator

    I’m convinced something about the import/export isn’t working right.

    It could be SELinux preventing it, could be firewall also.



  • Thanks for all your help guys! I really appreciate it!

    I went ahead and manually entered all my images info.

    I copied down all the settings in the fog settings page in preparation to enter it all manually.

    I sent one image successfully, just need to make sure hostname changer is working ok.

    I will work more on it Monday. Thanks again for all your help!


  • Testers

    Assuming that your original is still up and running, you could try doing a manual database dump and import. This is essentially what the gui is doing so that you don’t have to worry about the command line aspects, but it doesn’t

    In the command line on the original server, assuming you went with the default blank password…

    mysqldump -u root fog > /home/fog/fog.sql
    

    Then open up winscp, filezilla, cyberduck, or whatever ftp client you prefer (or just use scp in command line to copy straight to the vm) and download the fog.sql file you created.

    Then in the new vm server put the fog.sql file in /home/fog and then run this command

    mysql -u root fog < /home/fog/fog.sql
    

    And that should take care of it.

    Alternatively, especially if it looks like the gui export is working, you can just use the downloaded file from the gui for the second part there.

    Hope that helps


  • Moderator

    This is very, very strange everything should come across except for the physical media. I guess we need to check the apache error log. If you were on rhel it would be /var/log/httpd/error_log. Try the data import and then right away tail the error log and see if the import threw a silent error.



  • @george1421 I’ve restarted the server several times switching between the old and the new (same IP) so I don’t think that would be it.

    I did open the sql file yesterday and saw things that referred to my old install. I am just confused on what it does export and what it does not.

    Right now on my new install, after I import the database, I still get empty boxes in the Fog Configuration > Fog Settings > AD Defaults page.

    Is database import/export supposed to copy these settings?

    On a different note: Are you suggesting that I just need to record the images page from my old server and mimic these images in the new install? If I do this and the files are there then it should work I assume?


  • Moderator

    @JDnoble18 said:

    @jmeyer What do you mean by remaking images on GUI? Doesn’t import/export take care of this? I’ve already got the images copied over from the old server to the new one, do you suggest deleting the images folder and starting over?

    OK, just a second here. You did the export using the FOG configuration right. It should have downloaded a config file (fog_backup.sql) right? Of so open that with notepad++ and look at it. You will see database drops and creates, but there should also be insert into lines. This is placing your data back into the new FOG server. The images and snapin files are different but you should be able to copy the database no problem.

    … Just thinking. I wonder if you need to restart the fog service so that it sees the updated data? I’m not sure how much fog caches in memory.



  • @Wayne-Workman How stable is 1.3.0? Do you think upgrading my original machine would solve my issues when trying to get this new VM going?



  • @jmeyer What do you mean by remaking images on GUI? Doesn’t import/export take care of this? I’ve already got the images copied over from the old server to the new one, do you suggest deleting the images folder and starting over?


  • Moderator

    @JDnoble18 said:

    @george1421 I did this again, it said the import was successful…still no images, no hosts, and no ad settings. It looks like import/exporting might not work?

    What exactly does the import/export do?

    The import / export feature exportes the ENTIRE databsae - all settings, everything.
    the import overwrites all settings, putting in place everything from the import.


Log in to reply
 

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.