Locked out of Fog...

[george1421]

@msnyder Just for clarity you can’t login into the webgui or the linux console.

What user are you trying to login as?

If you can access the FOG server linux console, can you tail the apache error log to see if its throwing an error?

[msnyder]

Turns out I’m out of space…

I’m able to log into Bash, but can’t find the image to delete it…

/images/dev/???

[george1421]

@msnyder ok out of space is an issue for linux. Typically it will crash the OS, so if its still running you are lucky.

There shouldn’t be any images in /images/dev/<mac_address> If they are there then your captures have failed. You can delete the directories in /images/dev that are mac addresses only. There are other files in there that FOG needs to run.

You may have to purge an image directory out of /images/<image_name> to get enough free space for the OS to run. Once that is done then you have a few options. Remember that your images AND snapins are by default stored on the root partition. This is not something I like to do in a production environment, but that is the default for many linux distros.

1. In crease the size of the hard drive fog is installed on
2. Add a new hard drive (vmdk, vhd) to the FOG server and map that new disk over the /images directory after you copy the contents out.

[msnyder]

Interesting that a 9GB install turned into a 95GB image before it crashed…

How would one map /images to a newly added drive?

[george1421]

@msnyder You can start with this process: https://forums.fogproject.org/topic/6642/moving-fog-s-images-files-off-the-root-partition

Or the simpler way (if your fog server is on a virtual host server) is to just add a new disk to the virtual client.

Identify the new disk with lsblk
create a partition on the disk with fdisk /dev/sdX
Format the partition with mkfs.ext4 /dev/sdX1
mount the partition to a tmp directory like /mnt mount -t ext4 /dev/sdX1 /mnt
Move all of the files from /images to /mnt (that the new disk is mounted over) mv /images/* /mnt and mv /images/.* /mnt
Unmount the new disk umount /mnt
Update the fstab to mount /dev/sdX1 to /images.
Run the mount command to remount all of the partitions mount -a
Confirm it has been mounted df -h
Double check to make sure the directory permissions are still accurate

[msnyder]

@george1421 said in Locked out of Fog...:

mount -a

I think I’ve messed up the FSTAB:

root@PDC-FOG:/# mount -a
mount: special device /sdb1 does not exist
root@PDC-FOG:/# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0  100G  0 disk
├─sda1   8:1    0   96G  0 part /
├─sda2   8:2    0    1K  0 part
└─sda5   8:5    0    4G  0 part [SWAP]
sdb      8:16   0    2T  0 disk
└─sdb1   8:17   0    2T  0 part
sr0     11:0    1 1024M  0 rom

root@PDC-FOG:/# cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=d539e06c-4300-4f52-8a5f-b8b9585e8ec8 /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
UUID=46b75b5b-5cc6-474a-b0bf-9ac7afa2b49f none            swap    sw              0       0
/sdb1 /images ext3 defaults 0 2
root@PDC-FOG:/#

Did I do something wrong?

[msnyder]

@george1421 said in Locked out of Fog...:

df -h

I got it…

root@PDC-FOG:/# cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=d539e06c-4300-4f52-8a5f-b8b9585e8ec8 /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
UUID=46b75b5b-5cc6-474a-b0bf-9ac7afa2b49f none            swap    sw              0       0
/dev/sdb1       /images         ext4    defaults        0       2
root@PDC-FOG:/#

[george1421]

@msnyder Glad you have it worked out.

just confirm now in /images and /images/dev that you have the hidden file .mntcheck with ls -la /images and ls -la /images/dev and you should be all set.

I would also reboot the FOG server to make sure everything survives a reboot. That way 6 months from now you don’t wonder what happened to all of our images.

[msnyder]

Yep! Thanks for your help!

[msnyder]

I’ve captured my first image and it doesn’t seem to be replicating to my storage nodes… Could this be becasue I moved the /images to a new drive?

[george1421]

@msnyder no if the /images reconnected to /dev/sdb1 then the mount is good. Do the replicator logs in /opt/fog/log give you any idea?

[msnyder]

It appears that even though the capture completed on the machine, the image is staying in /images/dev/<macaddress>…

[george1421]

@msnyder Make sure your permissions are set correctly for /images

They should be owned by user fog:root right or wrong the file permissions are 777

[msnyder]

Correct?

root@PDC-FOG:/# ls -l
total 104
drwxr-xr-x   2 root root  4096 Jun 21 08:29 bin
drwxr-xr-x   3 root root  4096 Jun 21 08:25 boot
drwxr-xr-x  19 root root  3980 Jun 21 09:09 dev
drwxr-xr-x  98 root root  4096 Jun 21 15:25 etc
drwxr-xr-x   5 root root  4096 Jun 21 08:31 home
drwxr-xr-x   5 root root  4096 Jun 21 08:37 images
lrwxrwxrwx   1 root root    32 Jun 21 08:24 initrd.img -> boot/initrd.img-4.4.0-81-generic
lrwxrwxrwx   1 root root    32 May  9 14:37 initrd.img.old -> boot/initrd.img-4.4.0-77-generic
drwxr-xr-x  23 root root  4096 Jun 21 08:28 lib
drwxr-xr-x   3 root root  4096 May  9 14:50 lib32
drwxr-xr-x   2 root root  4096 Jun 21 08:22 lib64
drwx------   2 root root 16384 May  9 14:18 lost+found
drwxr-xr-x   3 root root  4096 May  9 14:18 media
drwxr-xr-x   3 root root  4096 May 22 09:22 mnt
drwxr-xr-x   4 root root  4096 Jun 21 08:32 opt
dr-xr-xr-x 239 root root     0 Jun 21 04:09 proc
drwx------   8 root root  4096 Jun 21 09:19 root
drwxr-xr-x  31 root root  1100 Jun 22 08:04 run
drwxr-xr-x   2 root root 12288 Jun 21 08:29 sbin
drwxr-xr-x   2 root root  4096 Jan 14 09:06 snap
drwxr-xr-x   3 root root  4096 Jun 21 08:31 srv
dr-xr-xr-x  13 root root     0 Jun 21 04:09 sys
drwxr-xr-x   6 fog  root  4096 Jun 21 08:33 tftpboot
drwxr-xr-x   2 root root  4096 Jun 21 08:33 tftpboot.prev
drwxrwxrwt  17 root root  4096 Jun 22 08:40 tmp
drwxr-xr-x  10 root root  4096 May  9 14:18 usr
drwxr-xr-x  14 root root  4096 Jun 21 08:28 var
lrwxrwxrwx   1 root root    29 Jun 21 08:24 vmlinuz -> boot/vmlinuz-4.4.0-81-generic
lrwxrwxrwx   1 root root    29 May  9 14:37 vmlinuz.old -> boot/vmlinuz-4.4.0-77-generic

root@PDC-FOG:/# cd /images/
root@PDC-FOG:/images# ls -l
total 24
drwxrwxrwx 5 fog  root  4096 Jun 21 14:01 dev
drwx------ 2 root root 16384 Jun 21 08:36 lost+found
drwxrwxrwx 2 fog  root  4096 Jun 21 08:33 postdownloadscripts

[Sebastian Roth]

@msnyder This should do the trick: chmod 777 /images

[msnyder]

@Sebastian-Roth said in Locked out of Fog...:

chmod 777 /images

Will I need to recapture my image now?

[george1421]

@msnyder You will need to capture the images again (or be sneeky and move the /images/dev/<mac_name> to /images/<image_name> as the <image_name> is exactly defined in the image configuration. mv /images/dev/<mac_name> /images/<image_name> The capture is complete it just needs to be parked in the right location.

[msnyder]

Seems to be replicating now! Thanks!