Lost Database

dospace

I attempted to update per the instructions here https://wiki.fogproject.org/wiki/index.php/Upgrade_to_trunk but the server was unable to connect to the database afterwards. I later figured out that the database was reinstalled. Is there a way to recover the old database or stored images?

Sebastian Roth

@dospace I can’t really imagine you can loose the database when simply following those update instructions. Which Linux do you use?

What makes you think that you completely lost your database?

dospace

Hey, thanks for your response. I’m using Ubuntu 18.04.2. Now that I’ve fixed the login I’ve found that all of the machines, groups, images, etc. that were there are gone. So I’m trying to figure out where it went and if there is a way to recover it. After the update I also found that MariaDB is installed so not sure if that changes things.

dospace

Sorry, I think that last response to myself. So far as I can tell the original fog database no longer exists. My main priority now is reattaching the images so that I don’t have to recreate them.

Sebastian Roth

@dospace Ok, let’s first focus on the image files. Please run ls -alR /images and du -sh /images to see if there is still something there.

Beside that please also run ls -alR /var/lib/mysql and ls -alR /var/lib/mariadb.

Copy and paste all the commands and output you get and post here.

dospace

Here is the output from that minus the vast amount of extra stuff from other directories:

ls -alR /images
.:
total 205
drwxrwxrwx  1 root root  8192 May 28 13:15 .
drwxr-xr-x 26 root root  4096 Jul  1 10:30 ..
drwxrwxrwx  1 root root  4096 Jul 24  2018 2018midpcs
drwxrwxrwx  1 root root  4096 Jun 29  2018 3DLab
drwxrwxrwx  1 root root  4096 Jul 15  2018 Allinones
drwxrwxrwx  1 root root  4096 May  1  2018 AutoCadPC
-rwxrwxrwx  1 root root    38 Jun 15  2015 Autorun.inf
drwxrwxrwx  1 root root     0 Jan 29  2018 bin
drwxrwxrwx  1 root root  4096 Apr 27  2018 BlueHPLaptops
drwxrwxrwx  1 root root  4096 May 28 13:15 dev
drwxrwxrwx  1 root root  4096 Jan 17  2017 Do Space Adobe Install PC
drwxrwxrwx  1 root root  4096 Dec  5  2018 DSLTux
drwxrwxrwx  1 root root  4096 Jul 24  2018 HighEndPCs
drwxrwxrwx  1 root root  4096 Jan 16  2017 HINT DoSpace_2017EssenceVideo_RawFootage
drwxrwxrwx  1 root root  4096 Mar 21  2018 Image001
drwxrwxrwx  1 root root  4096 Jun  2  2018 LaserCutter
drwxrwxrwx  1 root root  4096 Oct  9  2018 linuxsmall
-rwxrwxrwx  2 root root 44893 Nov  9  2017 material_doc.ico
drwxrwxrwx  1 root root  4096 Sep 12  2018 MidGreyBoxi5
-rwxrwxrwx  1 root root     0 Jan 29  2018 .mntcheck
drwxrwxrwx  1 root root  4096 Oct 31  2018 pc12
drwxrwxrwx  1 root root     0 Apr 26  2018 postdownloadscripts
drwxrwxrwx  1 root root  4096 Apr  5  2018 $RECYCLE.BIN
drwxrwxrwx  1 root root     0 Apr 24 13:51 rpimg
-rwxrwxrwx  1 root root 45514 Nov  9  2017 ruby_doc.ico
drwxrwxrwx  1 root root     0 Aug 10  2016 Seagate
drwxrwxrwx  1 root root     0 Apr  5  2018 Segate
drwxrwxrwx  1 root root  4096 May 28 13:15 student
drwxrwxrwx  1 root root  4096 Nov 11  2018 StudentPC
drwxrwxrwx  1 root root 12288 Jan 18  2017 Swanson Russel Files
drwxrwxrwx  1 root root  4096 Jan 24  2017 System Volume Information
drwxrwxrwx  1 root root  4096 Aug  9  2018 Temp
drwxrwxrwx  1 root root  4096 Nov 20  2018 WindowsLab
drwxrwxrwx  1 root root  4096 May  3  2018 XYZLaptops

./2018midpcs:
total 99032313
drwxrwxrwx 1 root root         4096 Jul 24  2018 .
drwxrwxrwx 1 root root         8192 May 28 13:15 ..
-rwxrwxrwx 1 root root      1048576 Jul 24  2018 d1.mbr
-rwxrwxrwx 1 root root    396207226 Jul 24  2018 d1p1.img
-rwxrwxrwx 1 root root 101011815812 Jul 24  2018 d1p2.img
-rwxrwxrwx 1 root root          190 Jul 24  2018 d1.partitions

./3DLab:
total 32356454
drwxrwxrwx 1 root root        4096 Jun 29  2018 .
drwxrwxrwx 1 root root        8192 May 28 13:15 ..
-rwxrwxrwx 1 root root           3 Jun 29  2018 d1.fixed_size_partitions
-rwxrwxrwx 1 root root     1048576 Jun 29  2018 d1.mbr
-rwxrwxrwx 1 root root         249 Jun 29  2018 d1.minimum.partitions
-rwxrwxrwx 1 root root          30 Jun 29  2018 d1.original.fstypes
-rwxrwxrwx 1 root root           0 Jun 29  2018 d1.original.swapuuids
-rwxrwxrwx 1 root root   385994221 Jun 29  2018 d1p1.img
-rwxrwxrwx 1 root root 32350207763 Jun 29  2018 d1p2.img
-rwxrwxrwx 1 root root   395742182 Jun 29  2018 d1p3.img
-rwxrwxrwx 1 root root         249 Jun 29  2018 d1.partitions

./Allinones:
total 165016977
drwxrwxrwx 1 root root         4096 Jul 15  2018 .
drwxrwxrwx 1 root root         8192 May 28 13:15 ..
-rwxrwxrwx 1 root root      1048576 Jul 15  2018 d1.mbr
-rwxrwxrwx 1 root root    397536896 Jul 15  2018 d1p1.img
-rwxrwxrwx 1 root root 168578784393 Jul 15  2018 d1p2.img
-rwxrwxrwx 1 root root          190 Jul 15  2018 d1.partitions

./DSLTux:
total 8518041
drwxrwxrwx 1 root root       4096 Dec  5  2018 .
drwxrwxrwx 1 root root       8192 May 28 13:15 ..
-rwxrwxrwx 1 root root          0 Dec  5  2018 d1.has_grub
-rwxrwxrwx 1 root root    1048576 Dec  5  2018 d1.mbr
-rwxrwxrwx 1 root root 8721411102 Dec  5  2018 d1p1.img
-rwxrwxrwx 1 root root        133 Dec  5  2018 d1.partitions

./HighEndPCs:
total 93028069
drwxrwxrwx 1 root root        4096 Jul 24  2018 .
drwxrwxrwx 1 root root        8192 May 28 13:15 ..
-rwxrwxrwx 1 root root     1048576 Jul 24  2018 d1.mbr
-rwxrwxrwx 1 root root   398012372 Jul 24  2018 d1p1.img
-rwxrwxrwx 1 root root 94861666536 Jul 24  2018 d1p2.img
-rwxrwxrwx 1 root root         190 Jul 24  2018 d1.partitions

./DSLTux:
total 8518041
drwxrwxrwx 1 root root       4096 Dec  5  2018 .
drwxrwxrwx 1 root root       8192 May 28 13:15 ..
-rwxrwxrwx 1 root root          0 Dec  5  2018 d1.has_grub
-rwxrwxrwx 1 root root    1048576 Dec  5  2018 d1.mbr
-rwxrwxrwx 1 root root 8721411102 Dec  5  2018 d1p1.img
-rwxrwxrwx 1 root root        133 Dec  5  2018 d1.partitions

./HighEndPCs:
total 93028069
drwxrwxrwx 1 root root        4096 Jul 24  2018 .
drwxrwxrwx 1 root root        8192 May 28 13:15 ..
-rwxrwxrwx 1 root root     1048576 Jul 24  2018 d1.mbr
-rwxrwxrwx 1 root root   398012372 Jul 24  2018 d1p1.img
-rwxrwxrwx 1 root root 94861666536 Jul 24  2018 d1p2.img
-rwxrwxrwx 1 root root         190 Jul 24  2018 d1.partitions

./student:
total 69458397
drwxrwxrwx 1 root root        4096 May 28 13:15 .
drwxrwxrwx 1 root root        8192 May 28 13:15 ..
-rwxrwxrwx 1 root root     1048576 May 28 12:43 d1.mbr
-rwxrwxrwx 1 root root    10357010 May 28 12:43 d1p1.img
-rwxrwxrwx 1 root root 71113974880 May 28 13:15 d1p2.img
-rwxrwxrwx 1 root root         204 May 28 12:43 d1.partitions

./StudentPC:
total 47078962
drwxrwxrwx 1 root root        4096 Nov 11  2018 .
drwxrwxrwx 1 root root        8192 May 28 13:15 ..
-rwxrwxrwx 1 root root           3 Nov 11  2018 d1.fixed_size_partitions
-rwxrwxrwx 1 root root     1048576 Nov 11  2018 d1.mbr
-rwxrwxrwx 1 root root         190 Nov 11  2018 d1.minimum.partitions
-rwxrwxrwx 1 root root          15 Nov 11  2018 d1.original.fstypes
-rwxrwxrwx 1 root root           0 Nov 11  2018 d1.original.swapuuids
-rwxrwxrwx 1 root root    10442210 Nov 11  2018 d1p1.img
-rwxrwxrwx 1 root root 48197346846 Nov 11  2018 d1p2.img
-rwxrwxrwx 1 root root         190 Nov 11  2018 d1.partitions

./WindowsLab:
total 69402458
drwxrwxrwx 1 root root        4096 Nov 20  2018 .
drwxrwxrwx 1 root root        8192 May 28 13:15 ..
-rwxrwxrwx 1 root root           3 Nov 20  2018 d1.fixed_size_partitions
-rwxrwxrwx 1 root root     1048576 Nov 20  2018 d1.mbr
-rwxrwxrwx 1 root root         190 Nov 20  2018 d1.minimum.partitions
-rwxrwxrwx 1 root root          15 Nov 20  2018 d1.original.fstypes
-rwxrwxrwx 1 root root           0 Nov 20  2018 d1.original.swapuuids
-rwxrwxrwx 1 root root    10442005 Nov 20  2018 d1p1.img
-rwxrwxrwx 1 root root 71056607057 Nov 20  2018 d1p2.img
-rwxrwxrwx 1 root root         190 Nov 20  2018 d1.partitions

./XYZLaptops:
total 16659986
drwxrwxrwx 1 root root        4096 May  3  2018 .
drwxrwxrwx 1 root root        8192 May 28 13:15 ..
-rwxrwxrwx 1 root root           3 May  3  2018 d1.fixed_size_partitions
-rwxrwxrwx 1 root root     1048576 May  3  2018 d1.mbr
-rwxrwxrwx 1 root root         249 May  3  2018 d1.minimum.partitions
-rwxrwxrwx 1 root root          30 May  3  2018 d1.original.fstypes
-rwxrwxrwx 1 root root           0 May  3  2018 d1.original.swapuuids
-rwxrwxrwx 1 root root   400726685 May  3  2018 d1p1.img
-rwxrwxrwx 1 root root 16268409135 May  3  2018 d1p2.img
-rwxrwxrwx 1 root root   389623530 May  3  2018 d1p3.img
-rwxrwxrwx 1 root root         249 May  3  2018 d1.partitions

du -sh /images
1.3T    /images/

ls -alR /var/lib/mysql
/var/lib/mysql:
total 110668
drwxr-xr-x  5 mysql mysql     4096 Jul  1 15:15 .
drwxr-xr-x 81 root  root      4096 Jun 30 16:35 ..
-rw-rw----  1 mysql mysql    16384 Jul  1 15:14 aria_log.00000001
-rw-rw----  1 mysql mysql       52 Jul  1 15:14 aria_log_control
-rw-r--r--  1 root  root         0 Jun 30 16:35 debian-10.1.flag
drwx------  2 mysql mysql    12288 Jul  2 09:27 fog
-rw-rw----  1 mysql mysql 12582912 Jul  1 15:14 ibdata1
-rw-rw----  1 mysql mysql 50331648 Jul  1 15:14 ib_logfile0
-rw-rw----  1 mysql mysql 50331648 Jun 30 16:35 ib_logfile1
-rw-rw----  1 mysql mysql        0 Jun 30 16:35 multi-master.info
drwx------  2 mysql mysql     4096 Jul  1 10:13 mysql
-rw-rw----  1 root  root        15 Jul  1 10:13 mysql_upgrade_info
drwx------  2 mysql mysql     4096 Jul  1 10:13 performance_schema
-rw-rw----  1 mysql mysql    24576 Jul  1 15:14 tc.log

/var/lib/mysql/fog:
total 3760
drwx------ 2 mysql mysql   12288 Jul  2 09:27 .
drwxr-xr-x 5 mysql mysql    4096 Jul  1 15:15 ..
-rw-rw---- 1 mysql mysql    3003 Jul  1 15:15 clientUpdates.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 clientUpdates.MYD
-rw-rw---- 1 mysql mysql    4096 Jul  1 15:15 clientUpdates.MYI
-rw-rw---- 1 mysql mysql      67 Jul  1 15:15 db.opt
-rw-rw---- 1 mysql mysql     959 Jul  1 15:15 dirCleaner.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 dirCleaner.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 dirCleaner.MYI
-rw-rw---- 1 mysql mysql    3318 Jul  1 15:15 globalSettings.frm
-rw-rw---- 1 mysql mysql   30944 Jul  1 15:15 globalSettings.MYD
-rw-rw---- 1 mysql mysql   57344 Jul  1 15:15 globalSettings.MYI
-rw-rw---- 1 mysql mysql    1630 Jul  1 15:15 greenFog.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 greenFog.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 greenFog.MYI
-rw-rw---- 1 mysql mysql    2940 Jul  1 15:15 groupMembers.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 groupMembers.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 groupMembers.MYI
-rw-rw---- 1 mysql mysql    5259 Jul  1 15:15 groups.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 groups.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 groups.MYI
-rw-rw---- 1 mysql mysql    3025 Jul  1 15:15 history.frm
-rw-rw---- 1 mysql mysql    2700 Jul  2 09:39 history.MYD
-rw-rw---- 1 mysql mysql    9216 Jul  2 09:39 history.MYI
-rw-rw---- 1 mysql mysql    2203 Jul  1 15:15 hookEvents.frm
-rw-rw---- 1 mysql mysql     888 Jul  2 09:39 hookEvents.MYD
-rw-rw---- 1 mysql mysql    9216 Jul  2 09:39 hookEvents.MYI
-rw-rw---- 1 mysql mysql    1992 Jul  1 15:15 hostAutoLogOut.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 hostAutoLogOut.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 hostAutoLogOut.MYI
-rw-rw---- 1 mysql mysql    3764 Jul  1 15:15 hostMAC.frm
-rw-rw---- 1 mysql mysql    3420 Jul  2 09:50 hostMAC.MYD
-rw-rw---- 1 mysql mysql   15360 Jul  2 09:50 hostMAC.MYI
-rw-rw---- 1 mysql mysql    2124 Jul  1 15:15 hostScreenSettings.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 hostScreenSettings.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 hostScreenSettings.MYI
-rw-rw---- 1 mysql mysql    8692 Jul  1 15:15 hosts.frm
-rw-rw---- 1 mysql mysql    2684 Jul  1 20:41 hosts.MYD
-rw-rw---- 1 mysql mysql    6144 Jul  2 13:19 hosts.MYI
-rw-rw---- 1 mysql mysql    2005 Jul  1 15:15 imageGroupAssoc.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 imageGroupAssoc.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 imageGroupAssoc.MYI
-rw-rw---- 1 mysql mysql    1841 Jul  1 15:15 imagePartitionTypes.frm
-rw-rw---- 1 mysql mysql     344 Jul  1 15:15 imagePartitionTypes.MYD
-rw-rw---- 1 mysql mysql    5120 Jul  1 15:15 imagePartitionTypes.MYI
-rw-rw---- 1 mysql mysql    4551 Jul  1 15:15 images.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 images.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 images.MYI
-rw-rw---- 1 mysql mysql    1814 Jul  1 15:15 imageTypes.frm
-rw-rw---- 1 mysql mysql     224 Jul  1 15:15 imageTypes.MYD
-rw-rw---- 1 mysql mysql    3072 Jul  1 15:15 imageTypes.MYI
-rw-rw---- 1 mysql mysql    2746 Jul  1 15:15 imagingLog.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 imagingLog.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 imagingLog.MYI
-rw-rw---- 1 mysql mysql   22811 Jul  1 15:15 inventory.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 inventory.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 inventory.MYI
-rw-rw---- 1 mysql mysql    1243 Jul  1 15:15 ipxeTable.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 ipxeTable.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 ipxeTable.MYI
-rw-rw---- 1 mysql mysql    1125 Jul  1 15:15 keySequence.frm
-rw-rw---- 1 mysql mysql     804 Jul  1 15:15 keySequence.MYD
-rw-rw---- 1 mysql mysql    2048 Jul  1 15:15 keySequence.MYI
-rw-rw---- 1 mysql mysql    1769 Jul  1 15:15 modules.frm
-rw-rw---- 1 mysql mysql    2716 Jul  1 15:15 modules.MYD
-rw-rw---- 1 mysql mysql    3072 Jul  1 15:15 modules.MYI
-rw-rw---- 1 mysql mysql    2970 Jul  1 15:15 moduleStatusByHost.frm
-rw-rw---- 1 mysql mysql    7540 Jul  1 20:41 moduleStatusByHost.MYD
-rw-rw---- 1 mysql mysql   34816 Jul  2 13:19 moduleStatusByHost.MYI
-rw-rw---- 1 mysql mysql    2442 Jul  1 15:15 multicastSessionsAssoc.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 multicastSessionsAssoc.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 multicastSessionsAssoc.MYI
-rw-rw---- 1 mysql mysql    5675 Jul  1 15:15 multicastSessions.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 multicastSessions.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 multicastSessions.MYI
-rw-rw---- 1 mysql mysql    4010 Jul  1 15:15 nfsFailures.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 nfsFailures.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 nfsFailures.MYI
-rw-rw---- 1 mysql mysql    8494 Jul  1 15:15 nfsGroupMembers.frm
-rw-rw---- 1 mysql mysql     236 Jul  1 15:15 nfsGroupMembers.MYD
-rw-rw---- 1 mysql mysql   20480 Jul  1 15:15 nfsGroupMembers.MYI
-rw-rw---- 1 mysql mysql    2713 Jul  1 15:15 nfsGroups.frm
-rw-rw---- 1 mysql mysql      48 Jul  1 15:15 nfsGroups.MYD
-rw-rw---- 1 mysql mysql   13312 Jul  1 15:15 nfsGroups.MYI
-rw-rw---- 1 mysql mysql    2203 Jul  1 15:15 notifyEvents.frm
-rw-rw---- 1 mysql mysql      20 Jul  1 15:15 notifyEvents.MYD
-rw-rw---- 1 mysql mysql    9216 Jul  1 15:15 notifyEvents.MYI
-rw-rw---- 1 mysql mysql    1567 Jul  1 15:15 os.frm
-rw-rw---- 1 mysql mysql     264 Jul  1 15:15 os.MYD
-rw-rw---- 1 mysql mysql    3072 Jul  1 15:15 os.MYI
-rw-rw---- 1 mysql mysql    2745 Jul  1 15:15 oui.frm
-rw-rw---- 1 mysql mysql 1095120 Jul  1 15:20 oui.MYD
-rw-rw---- 1 mysql mysql 1943552 Jul  1 15:20 oui.MYI
-rw-rw---- 1 mysql mysql    5708 Jul  1 15:15 plugins.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 plugins.MYD
-rw-rw---- 1 mysql mysql    2048 Jul  1 15:15 plugins.MYI
-rw-rw---- 1 mysql mysql    2399 Jul  1 15:15 powerManagement.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 powerManagement.MYD
-rw-rw---- 1 mysql mysql    4096 Jul  1 15:15 powerManagement.MYI
-rw-rw---- 1 mysql mysql    2649 Jul  1 15:15 printerAssoc.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 printerAssoc.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 printerAssoc.MYI
-rw-rw---- 1 mysql mysql    5204 Jul  1 15:15 printers.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 printers.MYD
-rw-rw---- 1 mysql mysql    4096 Jul  1 15:15 printers.MYI
-rw-rw---- 1 mysql mysql    3007 Jul  1 15:15 pxeMenu.frm
-rw-rw---- 1 mysql mysql    2484 Jul  1 15:15 pxeMenu.MYD
-rw-rw---- 1 mysql mysql    2048 Jul  1 15:15 pxeMenu.MYI
-rw-rw---- 1 mysql mysql    9510 Jul  1 15:15 scheduledTasks.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 scheduledTasks.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 scheduledTasks.MYI
-rw-rw---- 1 mysql mysql     950 Jul  1 15:15 schemaVersion.frm
-rw-rw---- 1 mysql mysql      20 Jul  1 15:15 schemaVersion.MYD
-rw-rw---- 1 mysql mysql    2048 Jul  1 15:15 schemaVersion.MYI
-rw-rw---- 1 mysql mysql    2941 Jul  1 15:15 snapinAssoc.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 snapinAssoc.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 snapinAssoc.MYI
-rw-rw---- 1 mysql mysql    2006 Jul  1 15:15 snapinGroupAssoc.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 snapinGroupAssoc.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 snapinGroupAssoc.MYI
-rw-rw---- 1 mysql mysql    1507 Jul  1 15:15 snapinJobs.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 snapinJobs.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 snapinJobs.MYI
-rw-rw---- 1 mysql mysql    5909 Jul  1 15:15 snapins.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 snapins.MYD
-rw-rw---- 1 mysql mysql    4096 Jul  1 15:15 snapins.MYI
-rw-rw---- 1 mysql mysql    3860 Jul  1 15:15 snapinTasks.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 snapinTasks.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 snapinTasks.MYI
-rw-rw---- 1 mysql mysql    2406 Jul  1 15:15 supportedOS.frm
-rw-rw---- 1 mysql mysql     196 Jul  1 15:15 supportedOS.MYD
-rw-rw---- 1 mysql mysql    6144 Jul  1 15:15 supportedOS.MYI
-rw-rw---- 1 mysql mysql    1203 Jul  1 15:15 taskLog.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 taskLog.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 taskLog.MYI
-rw-rw---- 1 mysql mysql   12768 Jul  1 15:15 tasks.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 tasks.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 tasks.MYI
-rw-rw---- 1 mysql mysql    2385 Jul  1 15:15 taskStates.frm
-rw-rw---- 1 mysql mysql     356 Jul  1 15:15 taskStates.MYD
-rw-rw---- 1 mysql mysql    3072 Jul  1 15:15 taskStates.MYI
-rw-rw---- 1 mysql mysql    2208 Jul  1 15:15 taskTypes.frm
-rw-rw---- 1 mysql mysql    4320 Jul  1 15:15 taskTypes.MYD
-rw-rw---- 1 mysql mysql    3072 Jul  1 15:15 taskTypes.MYI
-rw-rw---- 1 mysql mysql    1711 Jul  1 15:15 userCleanup.frm
-rw-rw---- 1 mysql mysql     128 Jul  1 15:15 userCleanup.MYD
-rw-rw---- 1 mysql mysql    2048 Jul  1 15:15 userCleanup.MYI
-rw-rw---- 1 mysql mysql    3420 Jul  1 15:15 users.frm
-rw-rw---- 1 mysql mysql     312 Jul  1 15:16 users.MYD
-rw-rw---- 1 mysql mysql    3072 Jul  1 15:16 users.MYI
-rw-rw---- 1 mysql mysql    3988 Jul  1 15:15 userTracking.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 userTracking.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 userTracking.MYI
-rw-rw---- 1 mysql mysql    3136 Jul  1 15:15 virus.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 virus.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 virus.MYI
-rw-rw---- 1 mysql mysql     312 Jul  1 15:16 users.MYD
-rw-rw---- 1 mysql mysql    3072 Jul  1 15:16 users.MYI
-rw-rw---- 1 mysql mysql    3988 Jul  1 15:15 userTracking.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 userTracking.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 userTracking.MYI
-rw-rw---- 1 mysql mysql    3136 Jul  1 15:15 virus.frm
-rw-rw---- 1 mysql mysql       0 Jul  1 15:15 virus.MYD
-rw-rw---- 1 mysql mysql    1024 Jul  1 15:15 virus.MYI

Sebastian Roth

@dospace Ok, your images are still there. Might be a good time to take a backup copy of the whole /images directory.

Now about the database. Kind of looks like the database files are still there and they have recent file change dates. Please try to manually connect to the database and see what’s in there.

shell> mysql -u root -p
Password:
...
mysql> use fog;
...
mysql> SELECT * FROM hosts;
...
mysql> SELECT * FROM images;
...

Please try this and post the output you get here in the forums.

dospace

To be clear, I have looked at the database and none of the things that were there are there anymore. The database files have recent dates because I’ve been working frantically to rebuild but it was new as of days ago when I tried to update. Don’t know what happened since I’ve never had that issue in previous updates. I was already aware that the images existed but not sure about the procedure to reattach them which is pretty much all I’ve got to figure out at this point. Thanks to the fog client most of my hosts are being redone without to much trouble. Images is an empty set. I did try to post the hosts output but it was to much.

Sebastian Roth

@dospace Ok, understand. Well re-creating the image definitions is possible and shouldn’t be too hard. Go to the web UI and click create new image. Now take a look at the file listings you have posted. Let’s start from the top. Name the new image 2018midpcs. OS type is important but you probably know which one that was set to!? Now as we only have the file d1.partitions but no d1.minimum.partitions we know this is a “Single disk - non-resizable” image type. Also an important setting. And last but not least compression algorithm and level. While level only plays a role if you re-capture the image the algorithm chosen here will also affect the tools FOG uses to extract the image files on deploy. So if the files are GZip compessed but you choose Zstd it will fail on deploy.

So just create one image after the other and play with the image settings till it deploys properly. Hope I gave you enough hints on what to pay attention to. Feel free to ask if one image doesn’t want to deploy properly.

Beside that I’d still be very interested in finding out what destroyed the database as we never had that before (I think) and the commands we use shouldn’t be able to cause such a situation. You say you use Ubuntu. Can you please post the full set of log files you find in /var/log/apt/* as well as /var/log/dpkg.log if existing. If you don’t want to share those in public you might upload the files to an online share and send me the link in a private message (see the speech bubble in the top right of the forum where I sent you a message already).

dospace

@Sebastian-Roth I can’t think of anything specific that should have caused and like I said I’ve never had it happen before during an update. If there’s anything that you can think of I’m interested. I’d rather not have this happen again for me or anyone else. I am going to setup a script to copy the database just in case though.

Sebastian Roth

We figured out that an upgrade from mysql to mariadb caused the issue. The database is not completely lost but more or less renamed. We found this hint in the apt logs:

The file /var/lib/mysql/debian-5.7.flag indicates a
version that cannot automatically be upgraded. Therefore the
previous data directory will be renamed to /var/lib/mysql-5.7 and
a new data directory will be initialized at /var/lib/mysql.
Please manually export/import your data (e.g. with mysqldump) if needed.

Seems like this is kind of a known problem: https://askubuntu.com/questions/766917/14-04-to-16-04-killed-mysql

Right now we are working on getting the old database back up. Not looking promising. MariaDB crashes terribly when we let it start on the old MySQL 5.7 DB files.

Edit: We were not able to start up the old DB with the new MariaDB version. So the only way I could see was setting up another system (probably old Ubuntu 14.04) with the old MySQL 5.7 version, get up the database there and do a mysqldump as this format can be imported into newer versions. So we decided that it’s probably less work to manually add the image definitions by hand than recovering the old DB. Hope this topic will be helpful for others running into this. Although I guess there aren’t many people still around upgrading from the old version.

@dospace I’ll mark this solved now as the initial issue is found. If you run into issues with manually creating the image definitions just open a new topic and post all the details there (ls -al /images/<IMAGENAME> as well as file /images/<IMAGENAME>/d1p1.img).

Lost Database

186

11.6k

17.1k

154.6k