Adding additional image storage space to FOG server

[george1421]

This tutorial will cover adding an additional hard drive to your FOG server to store more images. We will do this by formatting and mounting the new hard drive on the FOG server, and then in the FOG server configuration we will create a second storage node configuration on the FOG server
[editor note] I don’t care for your wording here

In this example I’ve added an additional vmdk (hard drive) to my testing FOG server. In the example below this new hard drive is connected to the fog server as /dev/sdb

( note1: for testing I also added a 3rd vmdk just in case I needed it later. That one is mounted as /dev/sdc. That hard drive will not be used in this tutorial so you may ignore it for the rest of this document )

( note2: my testing fog server’s OS is Centos 7, so the instructions are going to be Centos centric. You should be able to translate them to other linux distributions pretty easy with a little Google-fu )

1. First we’ll use lsblk to understand what block devices are connected to our fog server. You’ll notice that there are 2 “new” hard drives attached to my fog server without any partitions (sdb and sdc).

# lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0   30G  0 disk
├─sda1            8:1    0  500M  0 part /boot
└─sda2            8:2    0 29.5G  0 part
 ├─centos-root 253:0    0 26.5G  0 lvm  /
 └─centos-swap 253:1    0    3G  0 lvm  [SWAP]
sdb               8:16   0   40G  0 disk
sdc               8:32   0   50G  0 disk
sr0              11:0    1 1024M  0 rom

2. Now lets create a partition on /dev/sdb using the fdisk command. I’m just going to post the keystrokes needed to create the partition using fidsk
   fdisk /dev/sdb

And now the required keystrokes

n
p
1
<enter>
<enter>
w

The actual fdisk actions will look like this:

Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-83886079, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-83886079, default 83886079):
Using default value 83886079
Partition 1 of type Linux and of size 40 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
#

3. Use the lsblk command to confirm the partition is now visible on /dev/sdb
   lsblk

NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0   30G  0 disk
├─sda1            8:1    0  500M  0 part /boot
└─sda2            8:2    0 29.5G  0 part
  ├─centos-root 253:0    0 26.5G  0 lvm  /
  └─centos-swap 253:1    0    3G  0 lvm  [SWAP]
sdb               8:16   0   40G  0 disk
└─sdb1            8:17   0   40G  0 part
sdc               8:32   0   50G  0 disk
sr0              11:0    1 1024M  0 rom

4. Now lets format the partition (/dev/sdb1)
   mkfs.ext4 /dev/sdb1
   [Editor note: maybe should consider xfs file system instead of ext4, because xfs handles huge files better]

mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
2621440 inodes, 10485504 blocks
524275 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2157969408
320 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

5. Next we will create a new mount point (directory) to attach our hard drive partition (/dev/sdb1) to.
   mkdir /images2
6. Edit the fstab so our new drive is mounted to our mount point every time we reboot the fog server.
   vi /etc/fstab
7. Insert this line at the bottom of the fstab

/dev/sdb1 /images2 ext4 defaults 0 1

8. Use the df command to confirm the new drive is not connected (just to show a before and after example)
   df -h

Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/centos-root   27G  9.5G   18G  36% /
devtmpfs                 1.9G     0  1.9G   0% /dev
tmpfs                    1.9G     0  1.9G   0% /dev/shm
tmpfs                    1.9G  8.6M  1.9G   1% /run
tmpfs                    1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda1                497M  217M  281M  44% /boot
tmpfs                    380M     0  380M   0% /run/user/0

9. Notice that /dev/sdb does not appear in the printout above
10. Tell the file system to mount all devices in the fstab. I’m doing it this way to ensure when the FOG server reboots that the drives are mounted correctly. We could use the mount command directly such as in mount -t ext4 /dev/sdb1 /images2. But that wouldn’t guaranty that we keyed things in right in the fstab file.
    mount -a
11. Now repeat the df command
    df -h

Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/centos-root   27G  9.5G   18G  36% /
devtmpfs                 1.9G     0  1.9G   0% /dev
tmpfs                    1.9G     0  1.9G   0% /dev/shm
tmpfs                    1.9G  8.6M  1.9G   1% /run
tmpfs                    1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda1                497M  217M  281M  44% /boot
tmpfs                    380M     0  380M   0% /run/user/0
/dev/sdb1                 40G   49M   38G   1% /images2

12. Note that now we are mounting /dev/sdb1 to /images2
13. Lets create the required directory structure on our new drive

mkdir /images2/dev
mkdir /images2/dev/postinitscripts
mkdir /images2/postdownloadscripts

cp /images/dev/postinitscripts/* /images2/dev/postinitscripts
cp /images/postdownloadscripts/* /images2/postdownloadscripts
touch /images2/dev/.mntcheck
touch /images2/.mntcheck
chown -R fogproject.root /images2 

14. We need to update/create our nfs shares for our new disk. The FOS engine will mount these shares to capture and deploy images.
15. Use the showmount command to list the existing (current) shares
    showmount -e 127.0.0.1

Export list for 127.0.0.1:
/images/dev *
/images     *

16. You will notice that we have both /images and /images/dev shares currently shared. Now we will add our /image2 directories to the share list. We will do this by editing the exports file (similar to the fstab file, but for shared directories)
    vi /etc/exports
17. Append the following lines to the end of the /etc/exportfs file:
    Note: Ensure you update the fsid values if you copy and paste existing lines.

/images2 *(ro,sync,no_wdelay,no_subtree_check,insecure_locks,no_root_squash,insecure,fsid=3)
/images2/dev *(rw,async,no_wdelay,no_subtree_check,no_root_squash,insecure,fsid=4)

18. Save and exit the editor
19. Now tell the OS to reload the exports file.
    exportfs -ra
20. Rerun the showmount command to confirm we have our new drive shared
    showmount -e 127.0.0.1

Export list for 127.0.0.1:
/images2/dev *
/images2     *
/images/dev  *
/images      *

21. That concludes the operating system setup steps. Refer to Part 2 For the FOG server configuration to finish the setup.

[george1421]

Document place holder

[george1421]

Part 2 FOG WebGui Configuration

1. We need to create a new Storage Group. Go to Storage->Add Storage Group
2. Enter a new Storage Group Name and Description
3. Press Save/Update to create this new Storage Group.
   
4. Note that now we have 2 storage groups (default and ImageStoreGroup2)
   
5. Now we need to add a new Storage Node. Go to Storage->Add Storage Node.
   Enter the values in the storage node configuration appropriate for your setup. The key values to watch are the IP Address (fog server IP address), Is Master Node, Storage Group, Image and FTP Path. The rest can remain the defaults.
6. Its worth noting that the Management Username and Management Password are related to the FOG server where we just mounted to the new hard drive. If you don’t know what these values should be you can always inspect the default storage node settings. Just copy and paste them from the default storage node configuration to this new storage node configuration.
   
   
7. In the All Storage Nodes list note that the storage node ImageStore2 is attached to the ImageStoreGroup2 and that it is a Master Node in that storage group. This is important since only master nodes in a storage group can capture files.
   
8. Now we will setup an image definition to store images to this new storage group. Goto Image Management->Create New Image.
9. Enter the Image Name and select ImageStoreGroup2 for the Storage Group. Fill in the remaining fields as you would normally for the image.
   
10. Update your host to use this new image definition.
    
11. At this point we can schedule an image capture for this host. Go ahead an schedule a capture task for this host.
12. Below is a screen shot of the capture task as its running. The key thing to note here is that the Working with node points to our node definition that is pointing to /images2 directory. (Wh00t!!)
    
13. You can also confirm that the image is being saved to the /images2 directory by monitoring the partclone display.
    
14. We are done.

As you can see from this tutorial, its possible to add a second hard drive to your FOG server to add additional storage space by simply creating a loop back storage node definition. There are a number of steps involved with setting this up, so take it slow and follow each step exactly.