Feature request for FOG 1.6.x - Configure image capture to use NFSv4 instead of NFSv3
System security is becoming more of a concern and many compliance certifications will hit on the openness of the FOG imaging share. By switching over to NFSv4 we can use the additional security that is part of nfsv4 as well as reduce the number of open ports on the server to 1 for NFS. Reducing the number of port will allow FOG to integrate firewall rules in the the FOG design much easier than before.
On the client side this is what is needed to connect to an NFSv4 server on FOS Linux
mount -t nfs4 -o proto=tcp,port=2049,nolock,proto =tcp,rsize=32768,wsize=32768,intr,noatime "192.168.10.1:/" /images
We need to be mindful because mounts work a little differently on nfsv4 where the share point becomes the root of the share (akin to how MS Windows shares work) install of the full path. So sharing /images on the fog server the client would mount /images but the path would be in the root of /images path. Its just something FOG will need to take into account
@quinniedid Wow this has been 6 months already…
This thread really wasn’t intended to be a how-to document but more of an engineering working document to see if it is really possible to run nfsv4 with FOG. The intent was to minimize the number of open ports needed to make firewall rule crafting a bit easier. NFSv4 brings some interesting but confusing additions to the fog design. What I learned was that both the FOG server and FOS Linux needed some tweaks to get things working like with NFSv3.
@george1421 We are now being required to get NFSv4 utilized in our environment; have about 3 months before they shut off our server…
I have tried to follow some of the steps that you have taken to get this working. What I am confused about is where to make modifications to the FOS linux OS?
Would you be able to provide a little more details into the steps you took to get this working?
As I continue down this rabbit hole I find that in buildroot the nfs-utils in nfs-utils.mak nfsv4 is specifically disabled.
NFS_UTILS_CONF_OPTS = \ --disable-nfsv4 \ --disable-nfsv41 \ --disable-gss \ --disable-uuid \ --enable-tirpc \ --enable-ipv6 \ --without-tcp-wrappers \ --with-statedir=/run/nfs \ --with-rpcgen=$(HOST_DIR)/bin/rpcgen HOST_NFS_UTILS_CONF_OPTS = \ --disable-nfsv4 \ --disable-nfsv41 \ --disable-gss \ --disable-uuid \ --disable-ipv6 \ --without-tcp-wrappers \ --with-statedir=/run/nfs \ --disable-caps \ --disable-tirpc \ --without-systemd \
Need to be changed to enable nfsv4 to this
NFS_UTILS_CONF_OPTS = \ --enable-nfsv4 \ --enable-nfsv41 \ --disable-gss \ --disable-uuid \ --enable-tirpc \ --enable-ipv6 \ --without-tcp-wrappers \ --with-statedir=/run/nfs \ --with-rpcgen=$(HOST_DIR)/bin/rpcgen HOST_NFS_UTILS_CONF_OPTS = \ --enable-nfsv4 \ --enaable-nfsv41 \ --disable-gss \ --disable-uuid \ --disable-ipv6 \ --without-tcp-wrappers \ --with-statedir=/run/nfs \ --disable-caps \ --disable-tirpc \ --without-systemd \
Once enabled the package and initrd need to be recompiled.
In the initrd in /bin/fog.mount in lines 17 and 19
ref: mount -o nolock,proto=tcp,rsize=32768,wsize=32768,intr,noatime “$storage” /images >/tmp/mount-output 2>&1
The mount command needs to be updated to create an nfsv4 mount.
up) mount -o nolock,proto=tcp,rsize=32768,wsize=32768,intr,noatime "$storage" /images >/tmp/mount-output 2>&1 ;; down) mount -o nolock,proto=tcp,rsize=32768,intr,noatime "$storage" /images >/tmp/mount-output 2>&1 ;;
up) mount -o nolock,nfsvers=4,proto=tcp,rsize=32768,wsize=32768,intr,noatime "$storage" /images >/tmp/mount-output 2>&1 ;; down) mount -o nolock,nfsvers=4,proto=tcp,rsize=32768,intr,noatime "$storage" /images >/tmp/mount-output 2>&1 ;;
So far the FOG web code needs to be updated because the nfsv4 shares are presented differently than the >nfsv4.
nfsv3 share structure
nfsv4 shares structure
/ /images/ /capture/
./lib/fog/bootmenu.class.php starting at 1497 https://github.com/FOGProject/fogproject/blob/171d63724131c396029992730660497d48410842/packages/web/lib/fog/bootmenu.class.php#L1497
$storage = escapeshellcmd( sprintf( '%s:/%s/%s', $ip, trim($StorageNode->get('path'), '/'), ( $TaskType->isCapture() ? 'dev/' : '' ) ) );
$storage = escapeshellcmd( sprintf( '%s:/%s', $ip, ( $TaskType->isCapture() ? 'capture/' : trim($StorageNode->get('path'), '/') ) ) );
Its still not clear if there is a benefit to the cost of modifying the code to support NFS v4 over just keeping everything the same.
@george1421 I’ve worked with nfsv4 for about a week now. Many times I’ve stopped in irritation that the system doesn’t work like the examples but I’ve finally have a solid configuration for NFSv4. While nfsv4 is nfs, its very different in its setup than versions of NFS less than v4. NFSv4 creates a logical file system that doesn’t necessarily align with the physical file system on the server. In some ways it makes things more complex, in other ways it allows for some pretty cool redirection.
I’ve spent about 5 evenings trying to figure out how to map our current directory structure over to the nfsv4 file system. The problem was what ever the root was shared as ro or wr the child directories shares followed what the root share permissions were and ignored the specific share level permissions that the directories were exported as. I won’t go into details on the tests, but lets say I just ignored what I thought I knew and followed the examples and it worked.
Just to repeat it again, nfsv4 creates a logical file system with the root of the share, what you define as the root directory in your exportfs. You only assign the fsid=0 what you are calling the root of your logical file system. For my example I’ve created the logical file system structure under /opt/fog/data directory. In this example I’ve created bind mounts to the original FOG directories to maintain backwards compatibility.
Build the nfsv4 mount points
mkdir -p /opt/fog/data/capture mkdir -p /opt/fog/data/images
Freebie: so we present the same postinitscript files for both capture and imaging.
mkdir -p /images/dev/postinitscripts mkdir -p /images/postinitscripts
Create the bind loop back directories to map the physical file locations to the nfsv4 logical file system. Note this mapping can bring in directories from random locations but present them as a unified logical filesystem in NFS. This is the part that had me stuck for several days, where I thought I new better and this was unnecessary. But without this bind mount the child directories took on the share level permission of the logical nfs root.
-> /etc/fstab /images/dev /opt/fog/data/capture none bind 0 0 /images /opt/fog/data/images none bind 0 0 /images/postinitscripts /images/dev/postinitscripts none bind 0 0
Just a placeholder for when creating the fogproject user. I would recommend that the fog installer actually set a unique user and group IP when the fog project user is created. Then we can use only the squash root to assign the files created by the root user to the unique group and user ID. Its really only adding security through obscurity so I did not add it into the install instructions here.
We’ll add the fogproject user
Then we will grab the uid and gid of the fogproject user to use in the script that adds the nfs exports
$uid = `grep fogproject /etc/passwd | cut -d ":" -f3` $gid = `grep fogproject /etc/passwd | cut -d ":" -f4`
I’m only showing the product of the exports file not how $uid and $gid is set in the file
/opt/fog/data/ *(fsid=0,no_subtree_check,crossmnt,all_squash,insecure,anonuid=1000,anongid=1000) /opt/fog/data/capture *(rw,sync,no_subtree_check,no_wdelay,no_subtree_check,insecure_locks,all_squash,insecure,anonuid=1000,anongid=1000) /opt/fog/data/images *(ro,sync,no_subtree_check,all_squash,insecure,anonuid=1000,anongid=1000)
I understand the above is a big changed from what we have before. All users that connect to the nfs share will be squash all users to the fogproject user. The locks are only needed on a writeable share
To configure the nfs server to only honor nfsv4 requests the following file needs to be edited
[nfsd] tcp=y vers2=n vers3=n vers4=y vers4.0=y vers4.1=y vers4.2=y
Once the services are restarted the NFSv4 service will only accept nfs connects on tcp port 2049 where we can craft a custom firewall rule to protect without opening a large number of ports just for NFS.
There will be some changes needed in the FOS linux OS to accommodate this new NFSv4 structure. Most notably is to update the mount command to send the connect request
To capture an image
mount -t nfs -o nfsvers=4,rw 192.168.112.14:/capture /mnt/images
To deploy an image
mount -t nfs -o nfsvers=4,rw 192.168.112.14:/images /mnt/images
As a side note you can connect to the nfs root directory with this
mount -t nfs -o nfsvers=4,rw 192.168.112.14:/capture /mnt/images
And then move between the logical shares. The nfsv4 root share is exported as ro. When you change into the capture share with
cdthe share permissions automatically change to rw. If you
cdto /images the share permissions switch to ro. If you cd up one directory the change to ro.
This is what a you see when you execute these commands.
# mount -t nfs -o nfsvers=4,rw 192.168.112.14:/ /mnt/images # cd images/ # ls -la total 0 drwxr-xr-x. 4 root root 35 Dec 4 17:44 . drwxr-xr-x. 3 root root 20 Nov 30 18:49 .. drwxrwxrwx. 3 fogproject fogproject 85 Dec 4 17:52 capture drwxrwxrwx. 5 fogproject fogproject 82 Nov 29 12:05 images
Traversing these directory will change the share level permissions.
I’m still working on the ARM project so now I’ll update the FOS Linux client with this new file structure to see how well (if) it works.
There is a nice article on how to setup NFSv4 with a single port open: https://peteris.rocks/blog/nfs4-single-port/