Problem with external NFS on OpenMediaVault

[giolnl]

Hello everyone,

I’m trying FOG for my lab and I have it installed it on a VM in a proxmox cluster (so kvm virtualization), in the same cluster I already have an openmediavault server (10.21.100.29) and I was trying to use its NFS share as an alternative storage for FOG.

The problem is that when I try to upload an image of an host associated to that storage group it gave me this error:

mount: mounting 10.21.100.29:/images/dev/ on /images failed: Permission denied

On the OMV server log I get this:

openmediavault rpc.mountd[28433]: refused mount request from 10.21.100.12 for /images/dev/ (/): not exported

10.21.100.12 is the IP of the host I’m trying to imaging

what it boggles me is the fact that I’m able to mount that share without problem from both other linux clients and the FOG server itself

mount -t nfs 10.21.100.29:/images/ /mnt/images/

and

mount -t nfs 10.21.100.29:/images/dev /mnt/images/

both work well.

this is the output of ll -Ra /mnt/images/ in the FOG server after I mount the share (images)

/mnt/images/:
total 0
drwxrwxrwx  3 nobody nobody 32 30 lug 17.10 .
drwxr-xr-x. 3 root   root   19 30 lug 17.06 ..
drwxrwxrwx  2 nobody nobody 22 30 lug 17.10 dev
-rwxrwxrwx  1 nobody nobody  0 30 lug 17.10 .mntcheck

/mnt/images/dev:
total 0
drwxrwxrwx 2 nobody nobody 22 30 lug 17.10 .
drwxrwxrwx 3 nobody nobody 32 30 lug 17.10 ..
-rwxrwxrwx 1 nobody nobody  0 30 lug 17.10 .mntcheck

I already succesfully upload an image to a local storage.

Am I missing something?

Thanks

[Tom Elliott]

@giolnl Let me guess, you have the OpenMediaVault mounted to the FOG Server via NFS, then you’re attempting to NFS Mount the FOG Server’s /images (mounted to the NFS first) to the Clients. (That’s how the clients get the images is by NFS mounting the directory from the FOG Server).

The reason you’re getting permission denied is because An NFS Mount cannot be re-shared via NFS.

[giolnl]

Hi Tom, thanks for your reply!

Unless I misunderstood something I don’t think I’m in the situation you described (I already stumbled upon that case during some earlier tests though).

For clarity:
10.21.100.29: OMV Server
10.21.100.27: FOG Server

This is how i configured the new storage on FOG, so I’d expect that the NFS mount is done using the provided IP, which is the one of the OMV server, the fact that I can see the client trying to mount the NFS share in the OMV server log seems to prove that this is the case, but maybe I’m missing something.
The error message of the host also seems to point to that direction, it says that it fails to mount 10.21.100.29:/images/dev/ where 10.21.100.29 is indeed the IP address of the OMV server.

I also shared the same flder via FTP and created the fog user

It is possibile somehow to enter in a shell after booting via TFTP to make some tests?

[Tom Elliott]

The way you’re doing to NFS should work fine then. What does seem to be interesting is the fact that it’s stating the export for / is denied. This leads me to think two things. First the fsid parameter for the exports files does not contain a 0 identifier. I don’t know how one would configure the omv to use NFS though. The other possibility is maybe the omv setup is per ip rather than shared to everybody?

[giolnl]

Hi Tom, sorry for the late reply but I was busy with other projects (and holidays :P).
I tried to mess a bit around with the fsid option on the export but I didn’t manage to get it to work, the only thing I noticed is that if i force it to 0 (fsid=0) I can’t mount /images/dev/ and I get the same error I got when FOG try to do the same:

Command on the FOG server:

mount -t nfs 10.21.100.29:/images/dev/ /mnt/images/

Log on OMV:

Sep 28 12:28:43 omv rpc.mountd[4798]: /export and /export/images have same filehandle for *,10.21.0.0/16, using first
Sep 28 12:28:43 omv rpc.mountd[4798]: refused mount request from 10.21.100.27 for /images/dev/ (/): not exported

But if i mount only /images/ it works
I’m not sure how much that error message is really relevant anyway, it always say that when i try to mount a share that doesn’t exist.

EDIT: I think I should add the fact that the root directory exported by OMV is /export/ inside it there are the other shared folders (in my case images, so the real path is /export/images/) and I am able to mount it without problem, I guess the fsid option force to 0 is messing with this configuration.
Also the trailing slash of the Image Path in the storage node configuration is always removed (if I enter “/images/” and update it change it to “/images”, is this correct?

It is possibile to know what command is issued by the client when i try to mount the NFS share?
One other thing I noticed is that it seems that neither the web interface can access the server as the Disk Information in the dashboard is not working and it say to “Unable to pull server information!”

As for the second possibility the share was restricted to the 10.21.0.0/16 network, I made it open to everyone but with no luck, just to be sure I also tried to gave the IP address I use in the DHCP server (I use a fixed IP address bound to the MAC and change this as I needed because we don’t want a full working DHCP server on the network) and the manual mount works fine, the IP i use is 10.21.1.20 btw, so this shouldn’t be a problem in any case.

[Wayne Workman]

@giolnl What version of FOG are you using?

[giolnl]

@Wayne-Workman
FOG Server:

• FOG Version 1.2.0
• Distro: CentOS 7
• Kernel: 3.10.0-123.el7.x86_64
• nfs packages:

libnfsidmap.x86_64                      0.25-11.el7                    @base    
nfs-utils.x86_64                        1:1.3.0-0.8.el7                @base    

OpenMediaVault Server:

• OMV Version: 2.1.15 (last available)
• Distro: Debian 7.8
• Kernel: 3.2.68-1+deb7u2
• nfs packages:

ii  libnfsidmap2:amd64                 0.25-4                        amd64        NFS idmapping library
ii  nfs-common                         1:1.2.6-4                     amd64        NFS support files common to client and server
ii  nfs-kernel-server                  1:1.2.6-4                     amd64        support for NFS kernel server

[ch3i]

@giolnl Hi, can you post the content of your exports file ?

[giolnl]

@ch3i
Hi! this is the content of the export file in the OMV server:

# /etc/exports: the access control list for filesystems which may be exported
#               to NFS clients.  See exports(5).
/export/backup 10.21.0.0/16(rw,subtree_check,secure)
/export/images (rw,subtree_check,secure)

# NFSv4 - pseudo filesystem root
/export 10.21.0.0/16(ro,fsid=0,root_squash,no_subtree_check,hide)
/export (ro,fsid=0,root_squash,no_subtree_check,hide)

the first two exports are the ones configured by me using the web gui, the “pseudo filesystem root” part is hidden to the user and I never modified it during my tests

[ch3i]

@giolnl

Mine :

/fogimages *(ro,sync,no_wdelay,no_subtree_check,insecure_locks,no_root_squash,insecure,fsid=0)
/fogimages/dev *(rw,async,no_wdelay,no_subtree_check,no_root_squash,insecure,fsid=1)

Can you provide the storage configuration (a screenshot of webui) ?

[giolnl]

@ch3i
my export file on the FOG server looks like yours, some things I already tried:

• set the same options i found in the FOG server’s export to the OMV’s export and change/mess a bit with them
• use two separate exports for images and images/dev (always in OMV) with the exact options found in FOG (so something almost identical to the setup of the FOG server)

The storage configuration is the same i posted before: screenshot, where 10.21.100.29 is the OMV server

[Wayne Workman]

@giolnl said:

the FOG server looks like yours, some things I already tried:

You need an NFS Export for both the /images directory and the /images/dev directories - wherever those may be. For troubleshooting purposes, set world-writable permissions (777) on the /images directory just to see it get going without unnecessary complications.

[Sebastian Roth]

@giolnl I guess your /etc/exports on OMV should look like this:

/export/backup 10.21.0.0/16(rw,subtree_check,secure)
/export/images (rw,subtree_check,secure)
/export/images/dev (rw,subtree_check,secure)

# NFSv4 - pseudo filesystem root
/export 10.21.0.0/16(ro,fsid=0,root_squash,no_subtree_check,hide)
/export (ro,fsid=0,root_squash,no_subtree_check,hide)

Maybe even add the NFS parameters originally used by FOG (see ch3i’s post).

[Wayne Workman]

@Uncle-Frank I’d highly recommend that.

[giolnl]

@Uncle-Frank
I’ve just tested it with no success

EDIT: I noticed a new thing, all the other tests I’ve done I’ve always set the task from the web GUI and then start the client with PXE, this time I just started the client without any task pending and then tried to do a quick image and now I can see this error in the Image list in the GUI under “image size: ON SERVER”, don’t know if it can be usefull or not:

FOGFTP: Failed to connect. Host: 10.21.100.29, Error: Undefined property: FOGFTP::$link

END EDIT

/etc/exports on OMV:

# /etc/exports: the access control list for filesystems which may be exported
#               to NFS clients.  See exports(5).
/export/backup 10.21.0.0/16(rw,subtree_check,secure)
/export/images (rw,subtree_check,secure)
/export/dev (rw,subtree_check,secure)

# NFSv4 - pseudo filesystem root
/export 10.21.0.0/16(ro,fsid=0,root_squash,no_subtree_check,hide)
/export (ro,fsid=0,root_squash,no_subtree_check,hide)

Same errors:

Sep 30 05:48:43 omv rpc.mountd[18040]: refused mount request from 10.21.1.20 for /images/ (/): not exported

and permission denied on the client

root@omv:~# ll -aR /export/images/
/export/images/:
total 12
drwxrwxrwx+ 3 root   users   32 Jul 30 17:10 .
drwxr-xr-x  5 root   root  4096 Sep 30 08:35 ..
-rwxrwxrwx+ 1 nobody users    0 Jul 30 17:10 .mntcheck
drwxrwsrwx+ 2 nobody users   22 Jul 30 17:10 dev

/export/images/dev:
total 8
drwxrwsrwx+ 2 nobody users 22 Jul 30 17:10 .
drwxrwxrwx+ 3 root   users 32 Jul 30 17:10 ..
-rwxrwxrwx+ 1 nobody users  0 Jul 30 17:10 .mntcheck

[Sebastian Roth]

/export/images/dev not /export/dev I reckon

[Wayne Workman]

# /export/backup 10.21.0.0/16(rw,subtree_check,secure)
#Commented out the above line just for troubleshooting purposes.
/export/images *(ro,sync,no_wdelay,no_subtree_check,insecure_locks,no_root_squash,insecure,fsid=0)
/export/images/dev *(rw,async,no_wdelay,no_subtree_check,no_root_squash,insecure,fsid=1)

[giolnl]

@Uncle-Frank
The fact is that OMV doesn’t le me export “directly” a subdirectory, it let me create a shared folder which point to /export/images/dev/ but then it exports it as /export/dev/ which is linked somehow to /export/images/dev/ (I don’t know how to be honest),
Anyway I made some tests with a “forced” configuration (meaning I actually bypassed the GUI of OMV) and everything worked, so the problem is in the OMV configuration, I’m still doing some tests but i think the problem is related to some ACL you can set on those shared folders and theirs subdirectories.

I’ll be back when I’ll be sure to have a working “minimal” configuraton for both FOG and OMV.

Thanks anyone for pointing me to the right direction!

[giolnl]

Guess I was wrong about the ACL, in the end what made the difference was the fact that during the “forced” tests I used “/export/images” as image path in the storage configuration instead of “/images” (because of the missing “/export” lines in /etc/exports)

Here some screenshots with the working configuration I’m actually using:

OMV Shared folder configuration

OMV Shared folder ACL configuration

OMV NFS shares

FOG Storage configuration

The actual /etc/exports file on OMV:

# /etc/exports: the access control list for filesystems which may be exported
#               to NFS clients.  See exports(5).
/export/images (rw,subtree_check,secure)
/export/backup (rw,subtree_check,secure)

# NFSv4 - pseudo filesystem root
/export (ro,fsid=0,root_squash,no_subtree_check,hide)

Could be that NSFv4 pseudo fs to cause the problem? Maybe by giving FOG the full path I somehow bypass it, just a random guess tho, I don’t know how NFS works apart from the basics.

[Sebastian Roth]

With /export/images/dev missing in /etc/exports is it working now??