import or migrate images and host to a new serveur

patrice · Feb 28, 2021, 8:16 AM

Hello
I had a fog 1.5.8 server on ubuntu 18.04, but the system crashed, how can I restore the images and hosts to the new fog 1.5.9 (ubuntu 20.04) system?
I copied the / opt / fog and / images directories and executed chmod 777 –vR / images but the images and hosts do not appear on the fog interface.
thank you for your answers

Sebastian Roth · Feb 28, 2021, 6:00 PM

@patrice The image definitions (metadata) and hosts are all stored in a database. You’d need to get a dump of that database to be able to restore that information.

How badly crashed your old system? Can you still boot into it and get a dump of the database (using mysqldump command)? If not then there is a slight chance you might be able to copy the raw database files from the old system, setup a temporary Ubuntu 18.04 host to let it bring up the database to be able to pull a dump from it.

Database versions differ a fair bit between 18.04 (mariadb 10.1.47) and 20.04 (mariadb 10.3.25) and I could imagine it will fail if you just copy over the raw database from 18.04 to the new 20.04 system. On the other hand it’s worth a quick try. Here is a rough command list of how to do this. Don’t expect this to just work out of the box as I have not tested this and don’t know the state the crashed system is in! On the new 20.04 server you can try this if you have a clean copy of /var/lib/mysql from your old server.

systemctl stop mariadb
mv /var/lib/mysql /var/lib/mysql.fresh
# now copy /var/lib/mysql from the old 18.04 server to the new one
chown -R mysql:mysql /var/lib/mysql
systemctl start mariadb
systemctl status mariadb

Along with the database the passwords are now as they were in the old installation. So you will need to adjust /var/www/html/fog/lib/fog/config.class.php and /opt/fog/.fogsettings on your new system (find the passwords of your old install in that same files on your old server).

patrice · Feb 28, 2021, 8:31 PM

@sebastian-roth
Thank you very much for your answer.
The old system does not start anymore, when loading it becomes very slow and crashes at the level of maria DB

I think the SSD is broken
I will try to reinstall fog identically (ubuntu 18.04 and fog 1.5.8)
thank you for your reply.

Sebastian Roth · Feb 28, 2021, 2:52 PM

@patrice Do you get to the login prompt on the old server at some point? If yes, then run journalctl -u mariadb.service -b, take a picture and post here.

I will try to reinstall fog identically (ubuntu 18.04 and fog 1.5.8)

Be aware that the version of mariadb you seem to have on the old server (10.1.44 as we see in the picture) is not the same that you will have if you install 18.04 right now. Probably you didn’t install updates on that old system and still had an older version of mariadb compared to what is current in the Ubuntu repos for 18.04 (10.1.47).

patrice · Feb 28, 2021, 9:03 PM

@sebastian-roth
here are the screenshots

patrice · Feb 28, 2021, 9:20 PM

@sebastian-roth
I installed the server last year in January or February , the FOG station has never been updated.

george1421 · Mar 1, 2021, 12:26 AM

@patrice Is your fog server out of disk space? df -h

Also in /var/log/mariadb there is some log files that may give an idea why the sql server fails to start. But if you are out of space on your root partition that would also cause the database to start.

Sebastian Roth · Mar 1, 2021, 5:42 AM

@patrice Yes, check disk space first. Then take a look at log files in /var/log/mysql or /var/log/mariadb…

patrice · Mar 1, 2021, 1:03 PM

@george1421 @Sebastian-Roth
apologies, as I’m not the one using fog I didn’t think about looking at the disk space.
I was able to arrive on the graphical interface, but on firefox I cannot connect: Database connection unavailable
I think the database did not like :
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.

2021-03-01 19:01:48 140500859161728 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: The InnoDB memory heap is disabled
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: Using Linux native AIO
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: Using SSE crc32 instructions
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: Completed initialization of buffer pool
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: Highest supported file format is Barracuda.
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: 128 rollback segment(s) are active.
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: Waiting for purge to start
2021-03-01 19:01:48 140500859161728 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.46-86.2 started; log sequence number 1618255
2021-03-01 19:01:48 140500187215616 [Note] InnoDB: Dumping buffer pool(s) not yet started
2021-03-01 19:01:48 140500859161728 [Note] Plugin ‘FEEDBACK’ is disabled.
2021-03-01 19:01:48 140500859161728 [Note] Recovering after a crash using tc.log
2021-03-01 19:01:48 140500859161728 [ERROR] Can’t init tc log
2021-03-01 19:01:48 140500859161728 [ERROR] Aborting

george1421 · Mar 1, 2021, 7:29 PM

@patrice connect to the fog server linux console and run the command sudo df -h and post the results here.

patrice · Mar 1, 2021, 2:24 PM

@george1421

Voici le résulltat:

Sys. de fichiers Taille Utilisé Dispo Uti% Monté sur
udev               7,8G       0  7,8G   0% /dev
tmpfs              1,6G    1,9M  1,6G   1% /run
/dev/sda2          1,9T    1,7T   53G  98% /
tmpfs              7,8G     57M  7,8G   1% /dev/shm
tmpfs              5,0M    4,0K  5,0M   1% /run/lock
tmpfs              7,8G       0  7,8G   0% /sys/fs/cgroup
/dev/loop0         2,5M    2,5M     0 100% /snap/gnome-calculator/826
/dev/loop1         1,0M    1,0M     0 100% /snap/gnome-logs/100
/dev/loop2          98M     98M     0 100% /snap/core/10185
/dev/loop3         146M    146M     0 100% /snap/opera/99
/dev/loop4         163M    163M     0 100% /snap/gnome-3-28-1804/145
/dev/loop6          15M     15M     0 100% /snap/gnome-characters/399
/dev/loop5         3,8M    3,8M     0 100% /snap/gnome-system-monitor/127
/dev/loop8         4,3M    4,3M     0 100% /snap/gnome-calculator/544
/dev/loop7         146M    146M     0 100% /snap/opera/97
/dev/loop10         45M     45M     0 100% /snap/gtk-common-themes/1440
/dev/loop9          55M     55M     0 100% /snap/core18/1668
/dev/loop11        161M    161M     0 100% /snap/gnome-3-28-1804/116
/dev/loop12        1,0M    1,0M     0 100% /snap/gnome-logs/81
/dev/loop14         56M     56M     0 100% /snap/core18/1932
/dev/loop13        218M    218M     0 100% /snap/gnome-3-34-1804/60
/dev/loop15        2,3M    2,3M     0 100% /snap/gnome-system-monitor/148
/dev/loop16         33M     33M     0 100% /snap/chromium-ffmpeg/17
/dev/loop18         63M     63M     0 100% /snap/gtk-common-themes/1506
/dev/loop17         90M     90M     0 100% /snap/core/8268
/dev/loop19        384K    384K     0 100% /snap/gnome-characters/570
/dev/sda1          511M    6,1M  505M   2% /boot/efi
tmpfs              1,6G     40K  1,6G   1% /run/user/1000

george1421 · Mar 1, 2021, 2:28 PM

@patrice said in import or migrate images and host to a new serveur:

 /dev/sda2          1,9T    1,7T   53G  98% /

This will be a problem soon. You only have 53GB free. Look in /images/dev directory on the fog server linux console, for any directory names that look like mac addresses. If you are not currently uploading an image those directories should not exist. You can safely delete those bad upload directories to free some disk space.

But I have to say that 53GB is enough free space to allow mariadb to startup.

patrice · Mar 1, 2021, 8:30 PM

@george1421
Here is the result
I removed 3 disk images to restart the PC

george1421 · Mar 1, 2021, 2:33 PM

@patrice Now rereading the thread I see your old server crashed and this is the server where you are trying to get the mariadb running again. If you can’t get the old database server started you might have to rebuild the metadata by hand.

article on fixing corrupt innodb databases: https://chepri.com/mysql-innodb-corruption-and-recovery/

patrice · Mar 1, 2021, 8:38 PM

@george1421
I plugged the disk into usb and I had 0 bytes left. I then deleted 3 disk images and then I plugged the disk back into the computer, it started (no more problem with maria DB).
I then have in firefox: Database connection unavailable

patrice · Mar 1, 2021, 8:45 PM

@george1421
thank you for your answer george, I go through google translate that’s why I take time to answer
I will try to rebuild the database as you told me, tomorrow because suir I have to go to bed, I will work early tomorrow
I’ll keep you informed tomorrow
thank you

patrice · Mar 6, 2021, 8:32 AM

@george1421
Hello george,
I did not succeed in restoring the database, I do a complete reinstallation of the server.
Thank you for everything

import or migrate images and host to a new serveur

239

12.0k

17.3k

155.2k