Images on storage nodes deleted when hard drive on master fails
-
Hi, today we encountered an unpleasant (but quite logical) behavior of image sync between the master and storage node. Our FOG master has two separate disks for system and /images. The disk containing the images had failed and as a result, the /images folder where it mounts appeared empty. The sync service then thought that all images on the master node had been deleted so it deleted them on the storage node as well.
I’d like to suggest implementing a safety measure for the sync service preventing data loss - e.g. sync would not start if the /images/dev directory is not present in both master and storage nodes. -
Better yet,
Do NOT sync anything any further if the /images/.mntcheck and /images/dev/.mntcheck files do not exist.
Also, @aldorf , have you verified that the drive did indeed fail, or perhaps your /etc/fstab was modified and the drive simply did not mount?
-
@Wayne-Workman - FOG 1.2.0 running on Ubuntu 12.04 LTS. The drive did fail during normal operation - i.e. after running for several days, it got suddenly disconnected, I’m almost sure it’s a hardware issue (either the drive itself or the SATA controller). After reboot it mounts normally and the images resync on the other node.
