Locked out of Fog...

[msnyder]

Turns out I’m out of space…

I’m able to log into Bash, but can’t find the image to delete it…

/images/dev/???

[george1421]

@msnyder ok out of space is an issue for linux. Typically it will crash the OS, so if its still running you are lucky.

There shouldn’t be any images in /images/dev/<mac_address> If they are there then your captures have failed. You can delete the directories in /images/dev that are mac addresses only. There are other files in there that FOG needs to run.

You may have to purge an image directory out of /images/<image_name> to get enough free space for the OS to run. Once that is done then you have a few options. Remember that your images AND snapins are by default stored on the root partition. This is not something I like to do in a production environment, but that is the default for many linux distros.

1. In crease the size of the hard drive fog is installed on
2. Add a new hard drive (vmdk, vhd) to the FOG server and map that new disk over the /images directory after you copy the contents out.

[msnyder]

Interesting that a 9GB install turned into a 95GB image before it crashed…

How would one map /images to a newly added drive?

[george1421]

@msnyder You can start with this process: https://forums.fogproject.org/topic/6642/moving-fog-s-images-files-off-the-root-partition

Or the simpler way (if your fog server is on a virtual host server) is to just add a new disk to the virtual client.

Identify the new disk with lsblk
create a partition on the disk with fdisk /dev/sdX
Format the partition with mkfs.ext4 /dev/sdX1
mount the partition to a tmp directory like /mnt mount -t ext4 /dev/sdX1 /mnt
Move all of the files from /images to /mnt (that the new disk is mounted over) mv /images/* /mnt and mv /images/.* /mnt
Unmount the new disk umount /mnt
Update the fstab to mount /dev/sdX1 to /images.
Run the mount command to remount all of the partitions mount -a
Confirm it has been mounted df -h
Double check to make sure the directory permissions are still accurate

[msnyder]

@george1421 said in Locked out of Fog...:

mount -a

I think I’ve messed up the FSTAB:

root@PDC-FOG:/# mount -a
mount: special device /sdb1 does not exist
root@PDC-FOG:/# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0  100G  0 disk
├─sda1   8:1    0   96G  0 part /
├─sda2   8:2    0    1K  0 part
└─sda5   8:5    0    4G  0 part [SWAP]
sdb      8:16   0    2T  0 disk
└─sdb1   8:17   0    2T  0 part
sr0     11:0    1 1024M  0 rom

root@PDC-FOG:/# cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=d539e06c-4300-4f52-8a5f-b8b9585e8ec8 /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
UUID=46b75b5b-5cc6-474a-b0bf-9ac7afa2b49f none            swap    sw              0       0
/sdb1 /images ext3 defaults 0 2
root@PDC-FOG:/#

Did I do something wrong?

[msnyder]

@george1421 said in Locked out of Fog...:

df -h

I got it…

root@PDC-FOG:/# cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=d539e06c-4300-4f52-8a5f-b8b9585e8ec8 /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
UUID=46b75b5b-5cc6-474a-b0bf-9ac7afa2b49f none            swap    sw              0       0
/dev/sdb1       /images         ext4    defaults        0       2
root@PDC-FOG:/#

[george1421]

@msnyder Glad you have it worked out.

just confirm now in /images and /images/dev that you have the hidden file .mntcheck with ls -la /images and ls -la /images/dev and you should be all set.

I would also reboot the FOG server to make sure everything survives a reboot. That way 6 months from now you don’t wonder what happened to all of our images.

[msnyder]

Yep! Thanks for your help!

[msnyder]

I’ve captured my first image and it doesn’t seem to be replicating to my storage nodes… Could this be becasue I moved the /images to a new drive?

[george1421]

@msnyder no if the /images reconnected to /dev/sdb1 then the mount is good. Do the replicator logs in /opt/fog/log give you any idea?

[msnyder]

It appears that even though the capture completed on the machine, the image is staying in /images/dev/<macaddress>…

[george1421]

@msnyder Make sure your permissions are set correctly for /images

They should be owned by user fog:root right or wrong the file permissions are 777

[msnyder]

Correct?

root@PDC-FOG:/# ls -l
total 104
drwxr-xr-x   2 root root  4096 Jun 21 08:29 bin
drwxr-xr-x   3 root root  4096 Jun 21 08:25 boot
drwxr-xr-x  19 root root  3980 Jun 21 09:09 dev
drwxr-xr-x  98 root root  4096 Jun 21 15:25 etc
drwxr-xr-x   5 root root  4096 Jun 21 08:31 home
drwxr-xr-x   5 root root  4096 Jun 21 08:37 images
lrwxrwxrwx   1 root root    32 Jun 21 08:24 initrd.img -> boot/initrd.img-4.4.0-81-generic
lrwxrwxrwx   1 root root    32 May  9 14:37 initrd.img.old -> boot/initrd.img-4.4.0-77-generic
drwxr-xr-x  23 root root  4096 Jun 21 08:28 lib
drwxr-xr-x   3 root root  4096 May  9 14:50 lib32
drwxr-xr-x   2 root root  4096 Jun 21 08:22 lib64
drwx------   2 root root 16384 May  9 14:18 lost+found
drwxr-xr-x   3 root root  4096 May  9 14:18 media
drwxr-xr-x   3 root root  4096 May 22 09:22 mnt
drwxr-xr-x   4 root root  4096 Jun 21 08:32 opt
dr-xr-xr-x 239 root root     0 Jun 21 04:09 proc
drwx------   8 root root  4096 Jun 21 09:19 root
drwxr-xr-x  31 root root  1100 Jun 22 08:04 run
drwxr-xr-x   2 root root 12288 Jun 21 08:29 sbin
drwxr-xr-x   2 root root  4096 Jan 14 09:06 snap
drwxr-xr-x   3 root root  4096 Jun 21 08:31 srv
dr-xr-xr-x  13 root root     0 Jun 21 04:09 sys
drwxr-xr-x   6 fog  root  4096 Jun 21 08:33 tftpboot
drwxr-xr-x   2 root root  4096 Jun 21 08:33 tftpboot.prev
drwxrwxrwt  17 root root  4096 Jun 22 08:40 tmp
drwxr-xr-x  10 root root  4096 May  9 14:18 usr
drwxr-xr-x  14 root root  4096 Jun 21 08:28 var
lrwxrwxrwx   1 root root    29 Jun 21 08:24 vmlinuz -> boot/vmlinuz-4.4.0-81-generic
lrwxrwxrwx   1 root root    29 May  9 14:37 vmlinuz.old -> boot/vmlinuz-4.4.0-77-generic

root@PDC-FOG:/# cd /images/
root@PDC-FOG:/images# ls -l
total 24
drwxrwxrwx 5 fog  root  4096 Jun 21 14:01 dev
drwx------ 2 root root 16384 Jun 21 08:36 lost+found
drwxrwxrwx 2 fog  root  4096 Jun 21 08:33 postdownloadscripts

[Sebastian Roth]

@msnyder This should do the trick: chmod 777 /images

[msnyder]

@Sebastian-Roth said in Locked out of Fog...:

chmod 777 /images

Will I need to recapture my image now?

[george1421]

@msnyder You will need to capture the images again (or be sneeky and move the /images/dev/<mac_name> to /images/<image_name> as the <image_name> is exactly defined in the image configuration. mv /images/dev/<mac_name> /images/<image_name> The capture is complete it just needs to be parked in the right location.

[msnyder]

Seems to be replicating now! Thanks!