Feature request for FOG 1.6.x - Configure image capture to use NFSv4 instead of NFSv3

  • Moderator

    System security is becoming more of a concern and many compliance certifications will hit on the openness of the FOG imaging share. By switching over to NFSv4 we can use the additional security that is part of nfsv4 as well as reduce the number of open ports on the server to 1 for NFS. Reducing the number of port will allow FOG to integrate firewall rules in the the FOG design much easier than before.

    On the client side this is what is needed to connect to an NFSv4 server on FOS Linux
    mount -t nfs4 -o proto=tcp,port=2049,nolock,proto =tcp,rsize=32768,wsize=32768,intr,noatime "192.168.10.1:/" /images

    We need to be mindful because mounts work a little differently on nfsv4 where the share point becomes the root of the share (akin to how MS Windows shares work) install of the full path. So sharing /images on the fog server the client would mount /images but the path would be in the root of /images path. Its just something FOG will need to take into account

  • Moderator

    As I continue down this rabbit hole I find that in buildroot the nfs-utils in nfs-utils.mak nfsv4 is specifically disabled.

    NFS_UTILS_CONF_OPTS = \
            --disable-nfsv4 \
            --disable-nfsv41 \
            --disable-gss \
            --disable-uuid \
            --enable-tirpc \
            --enable-ipv6 \
            --without-tcp-wrappers \
            --with-statedir=/run/nfs \
            --with-rpcgen=$(HOST_DIR)/bin/rpcgen
    
    HOST_NFS_UTILS_CONF_OPTS = \
            --disable-nfsv4 \
            --disable-nfsv41 \
            --disable-gss \
            --disable-uuid \
            --disable-ipv6 \
            --without-tcp-wrappers \
            --with-statedir=/run/nfs \
            --disable-caps \
            --disable-tirpc \
            --without-systemd \
    

    Need to be changed to enable nfsv4 to this

    NFS_UTILS_CONF_OPTS = \
            --enable-nfsv4 \
            --enable-nfsv41 \
            --disable-gss \
            --disable-uuid \
            --enable-tirpc \
            --enable-ipv6 \
            --without-tcp-wrappers \
            --with-statedir=/run/nfs \
            --with-rpcgen=$(HOST_DIR)/bin/rpcgen
    
    HOST_NFS_UTILS_CONF_OPTS = \
            --enable-nfsv4 \
            --enaable-nfsv41 \
            --disable-gss \
            --disable-uuid \
            --disable-ipv6 \
            --without-tcp-wrappers \
            --with-statedir=/run/nfs \
            --disable-caps \
            --disable-tirpc \
            --without-systemd \
    
    

    Once enabled the package and initrd need to be recompiled.

    In the initrd in /bin/fog.mount in lines 17 and 19
    ref: mount -o nolock,proto=tcp,rsize=32768,wsize=32768,intr,noatime “$storage” /images >/tmp/mount-output 2>&1

    The mount command needs to be updated to create an nfsv4 mount.
    from this

        up)
            mount -o nolock,proto=tcp,rsize=32768,wsize=32768,intr,noatime "$storage" /images >/tmp/mount-output 2>&1
            ;;
        down)
            mount -o nolock,proto=tcp,rsize=32768,intr,noatime "$storage" /images >/tmp/mount-output 2>&1
            ;;
    

    to this

        up)
            mount -o nolock,nfsvers=4,proto=tcp,rsize=32768,wsize=32768,intr,noatime "$storage" /images >/tmp/mount-output 2>&1
            ;;
        down)
            mount -o nolock,nfsvers=4,proto=tcp,rsize=32768,intr,noatime "$storage" /images >/tmp/mount-output 2>&1
            ;;
    

    So far the FOG web code needs to be updated because the nfsv4 shares are presented differently than the >nfsv4.

    nfsv3 share structure

    /images/
    /images/dev/
    

    nfsv4 shares structure

    /
    /images/
    /capture/
    

    ./lib/fog/bootmenu.class.php starting at 1497 https://github.com/FOGProject/fogproject/blob/171d63724131c396029992730660497d48410842/packages/web/lib/fog/bootmenu.class.php#L1497

    Replacing

                        $storage = escapeshellcmd(
                            sprintf(
                                '%s:/%s/%s',
                                $ip,
                                trim($StorageNode->get('path'), '/'),
                                (
                                    $TaskType->isCapture() ?
                                    'dev/' :
                                    ''
                                )
                            )
                        );
    

    to this

                        $storage = escapeshellcmd(
                             sprintf(
                                 '%s:/%s',
                                 $ip,
                                 (
                                     $TaskType->isCapture() ?
                                     'capture/' :
                                     trim($StorageNode->get('path'), '/')
                                 )
                             )
                         ); 
    

    Its still not clear if there is a benefit to the cost of modifying the code to support NFS v4 over just keeping everything the same.

  • Moderator

    @george1421 I’ve worked with nfsv4 for about a week now. Many times I’ve stopped in irritation that the system doesn’t work like the examples but I’ve finally have a solid configuration for NFSv4. While nfsv4 is nfs, its very different in its setup than versions of NFS less than v4. NFSv4 creates a logical file system that doesn’t necessarily align with the physical file system on the server. In some ways it makes things more complex, in other ways it allows for some pretty cool redirection.

    I’ve spent about 5 evenings trying to figure out how to map our current directory structure over to the nfsv4 file system. The problem was what ever the root was shared as ro or wr the child directories shares followed what the root share permissions were and ignored the specific share level permissions that the directories were exported as. I won’t go into details on the tests, but lets say I just ignored what I thought I knew and followed the examples and it worked.
    ref: https://cwiki.apache.org/confluence/display/DIRxINTEROP/NFSv4+Server+Export+Setup

    Just to repeat it again, nfsv4 creates a logical file system with the root of the share, what you define as the root directory in your exportfs. You only assign the fsid=0 what you are calling the root of your logical file system. For my example I’ve created the logical file system structure under /opt/fog/data directory. In this example I’ve created bind mounts to the original FOG directories to maintain backwards compatibility.

    Build the nfsv4 mount points

    mkdir -p /opt/fog/data/capture
    mkdir -p /opt/fog/data/images
    

    Freebie: so we present the same postinitscript files for both capture and imaging.

    mkdir -p /images/dev/postinitscripts
    mkdir -p /images/postinitscripts
    

    Create the bind loop back directories to map the physical file locations to the nfsv4 logical file system. Note this mapping can bring in directories from random locations but present them as a unified logical filesystem in NFS. This is the part that had me stuck for several days, where I thought I new better and this was unnecessary. But without this bind mount the child directories took on the share level permission of the logical nfs root.

    -> /etc/fstab
    /images/dev    /opt/fog/data/capture            none    bind 0 0
    /images        /opt/fog/data/images             none    bind 0 0
    /images/postinitscripts  /images/dev/postinitscripts   none    bind 0 0
    

    Just a placeholder for when creating the fogproject user. I would recommend that the fog installer actually set a unique user and group IP when the fog project user is created. Then we can use only the squash root to assign the files created by the root user to the unique group and user ID. Its really only adding security through obscurity so I did not add it into the install instructions here.

    We’ll add the fogproject user

    adduser fogproject
    

    Then we will grab the uid and gid of the fogproject user to use in the script that adds the nfs exports

    $uid = `grep fogproject /etc/passwd | cut -d ":" -f3`
    $gid = `grep fogproject /etc/passwd | cut -d ":" -f4`
    

    I’m only showing the product of the exports file not how $uid and $gid is set in the file
    -> /etc/exports

    /opt/fog/data/ *(fsid=0,no_subtree_check,crossmnt,all_squash,insecure,anonuid=1000,anongid=1000)
    /opt/fog/data/capture *(rw,sync,no_subtree_check,no_wdelay,no_subtree_check,insecure_locks,all_squash,insecure,anonuid=1000,anongid=1000)
    /opt/fog/data/images *(ro,sync,no_subtree_check,all_squash,insecure,anonuid=1000,anongid=1000)
    

    I understand the above is a big changed from what we have before. All users that connect to the nfs share will be squash all users to the fogproject user. The locks are only needed on a writeable share

    To configure the nfs server to only honor nfsv4 requests the following file needs to be edited
    -> /etc/nfs.conf

    [nfsd]
    tcp=y
    vers2=n
    vers3=n
    vers4=y
    vers4.0=y
    vers4.1=y
    vers4.2=y
    

    Once the services are restarted the NFSv4 service will only accept nfs connects on tcp port 2049 where we can craft a custom firewall rule to protect without opening a large number of ports just for NFS.

    There will be some changes needed in the FOS linux OS to accommodate this new NFSv4 structure. Most notably is to update the mount command to send the connect request

    To capture an image

    mount -t nfs -o nfsvers=4,rw 192.168.112.14:/capture /mnt/images
    
    

    To deploy an image

    mount -t nfs -o nfsvers=4,rw 192.168.112.14:/images /mnt/images
    

    As a side note you can connect to the nfs root directory with this

    mount -t nfs -o nfsvers=4,rw 192.168.112.14:/capture /mnt/images
    

    And then move between the logical shares. The nfsv4 root share is exported as ro. When you change into the capture share with cd the share permissions automatically change to rw. If you cd to /images the share permissions switch to ro. If you cd up one directory the change to ro.

    This is what a you see when you execute these commands.

    # mount -t nfs -o nfsvers=4,rw 192.168.112.14:/ /mnt/images
    # cd images/
    # ls -la
    total 0
    drwxr-xr-x. 4 root  root 35 Dec  4 17:44 .
    drwxr-xr-x. 3 root  root 20 Nov 30 18:49 ..
    drwxrwxrwx. 3 fogproject fogproject 85 Dec  4 17:52 capture
    drwxrwxrwx. 5 fogproject fogproject 82 Nov 29 12:05 images
    

    Traversing these directory will change the share level permissions.

    I’m still working on the ARM project so now I’ll update the FOS Linux client with this new file structure to see how well (if) it works.

  • Senior Developer

    There is a nice article on how to setup NFSv4 with a single port open: https://peteris.rocks/blog/nfs4-single-port/

343
Online

8.3k
Users

15.1k
Topics

142.0k
Posts