Problem with external NFS on OpenMediaVault
-
Hello everyone,
I’m trying FOG for my lab and I have it installed it on a VM in a proxmox cluster (so kvm virtualization), in the same cluster I already have an openmediavault server (10.21.100.29) and I was trying to use its NFS share as an alternative storage for FOG.
The problem is that when I try to upload an image of an host associated to that storage group it gave me this error:
mount: mounting 10.21.100.29:/images/dev/ on /images failed: Permission deniedOn the OMV server log I get this:
openmediavault rpc.mountd[28433]: refused mount request from 10.21.100.12 for /images/dev/ (/): not exported10.21.100.12 is the IP of the host I’m trying to imaging
what it boggles me is the fact that I’m able to mount that share without problem from both other linux clients and the FOG server itself
mount -t nfs 10.21.100.29:/images/ /mnt/images/and
mount -t nfs 10.21.100.29:/images/dev /mnt/images/both work well.
this is the output of ll -Ra /mnt/images/ in the FOG server after I mount the share (images)
/mnt/images/: total 0 drwxrwxrwx 3 nobody nobody 32 30 lug 17.10 . drwxr-xr-x. 3 root root 19 30 lug 17.06 .. drwxrwxrwx 2 nobody nobody 22 30 lug 17.10 dev -rwxrwxrwx 1 nobody nobody 0 30 lug 17.10 .mntcheck /mnt/images/dev: total 0 drwxrwxrwx 2 nobody nobody 22 30 lug 17.10 . drwxrwxrwx 3 nobody nobody 32 30 lug 17.10 .. -rwxrwxrwx 1 nobody nobody 0 30 lug 17.10 .mntcheckI already succesfully upload an image to a local storage.
Am I missing something?
Thanks
-
@giolnl Let me guess, you have the OpenMediaVault mounted to the FOG Server via NFS, then you’re attempting to NFS Mount the FOG Server’s /images (mounted to the NFS first) to the Clients. (That’s how the clients get the images is by NFS mounting the directory from the FOG Server).
The reason you’re getting permission denied is because An NFS Mount cannot be re-shared via NFS.
-
Hi Tom, thanks for your reply!
Unless I misunderstood something I don’t think I’m in the situation you described (I already stumbled upon that case during some earlier tests though).
For clarity:
10.21.100.29: OMV Server
10.21.100.27: FOG ServerThis is how i configured the new storage on FOG, so I’d expect that the NFS mount is done using the provided IP, which is the one of the OMV server, the fact that I can see the client trying to mount the NFS share in the OMV server log seems to prove that this is the case, but maybe I’m missing something.
The error message of the host also seems to point to that direction, it says that it fails to mount 10.21.100.29:/images/dev/ where 10.21.100.29 is indeed the IP address of the OMV server.I also shared the same flder via FTP and created the fog user
It is possibile somehow to enter in a shell after booting via TFTP to make some tests?
-
The way you’re doing to NFS should work fine then. What does seem to be interesting is the fact that it’s stating the export for / is denied. This leads me to think two things. First the fsid parameter for the exports files does not contain a 0 identifier. I don’t know how one would configure the omv to use NFS though. The other possibility is maybe the omv setup is per ip rather than shared to everybody?
-
Hi Tom, sorry for the late reply but I was busy with other projects (and holidays :P).
I tried to mess a bit around with the fsid option on the export but I didn’t manage to get it to work, the only thing I noticed is that if i force it to 0 (fsid=0) I can’t mount /images/dev/ and I get the same error I got when FOG try to do the same:Command on the FOG server:
mount -t nfs 10.21.100.29:/images/dev/ /mnt/images/Log on OMV:
Sep 28 12:28:43 omv rpc.mountd[4798]: /export and /export/images have same filehandle for *,10.21.0.0/16, using first Sep 28 12:28:43 omv rpc.mountd[4798]: refused mount request from 10.21.100.27 for /images/dev/ (/): not exportedBut if i mount only /images/ it works
I’m not sure how much that error message is really relevant anyway, it always say that when i try to mount a share that doesn’t exist.EDIT: I think I should add the fact that the root directory exported by OMV is /export/ inside it there are the other shared folders (in my case images, so the real path is /export/images/) and I am able to mount it without problem, I guess the fsid option force to 0 is messing with this configuration.
Also the trailing slash of the Image Path in the storage node configuration is always removed (if I enter “/images/” and update it change it to “/images”, is this correct?It is possibile to know what command is issued by the client when i try to mount the NFS share?
One other thing I noticed is that it seems that neither the web interface can access the server as the Disk Information in the dashboard is not working and it say to “Unable to pull server information!”As for the second possibility the share was restricted to the 10.21.0.0/16 network, I made it open to everyone but with no luck, just to be sure I also tried to gave the IP address I use in the DHCP server (I use a fixed IP address bound to the MAC and change this as I needed because we don’t want a full working DHCP server on the network) and the manual mount works fine, the IP i use is 10.21.1.20 btw, so this shouldn’t be a problem in any case.
-
@giolnl What version of FOG are you using?
-
@Wayne-Workman
FOG Server:- FOG Version 1.2.0
- Distro: CentOS 7
- Kernel: 3.10.0-123.el7.x86_64
- nfs packages:
libnfsidmap.x86_64 0.25-11.el7 @base nfs-utils.x86_64 1:1.3.0-0.8.el7 @baseOpenMediaVault Server:
- OMV Version: 2.1.15 (last available)
- Distro: Debian 7.8
- Kernel: 3.2.68-1+deb7u2
- nfs packages:
ii libnfsidmap2:amd64 0.25-4 amd64 NFS idmapping library ii nfs-common 1:1.2.6-4 amd64 NFS support files common to client and server ii nfs-kernel-server 1:1.2.6-4 amd64 support for NFS kernel server -
@giolnl Hi, can you post the content of your
exportsfile ? -
@ch3i
Hi! this is the content of the export file in the OMV server:# /etc/exports: the access control list for filesystems which may be exported # to NFS clients. See exports(5). /export/backup 10.21.0.0/16(rw,subtree_check,secure) /export/images (rw,subtree_check,secure) # NFSv4 - pseudo filesystem root /export 10.21.0.0/16(ro,fsid=0,root_squash,no_subtree_check,hide) /export (ro,fsid=0,root_squash,no_subtree_check,hide)the first two exports are the ones configured by me using the web gui, the “pseudo filesystem root” part is hidden to the user and I never modified it during my tests
-
Mine :
/fogimages *(ro,sync,no_wdelay,no_subtree_check,insecure_locks,no_root_squash,insecure,fsid=0) /fogimages/dev *(rw,async,no_wdelay,no_subtree_check,no_root_squash,insecure,fsid=1)Can you provide the storage configuration (a screenshot of webui) ?
-
@ch3i
my export file on the FOG server looks like yours, some things I already tried:- set the same options i found in the FOG server’s export to the OMV’s export and change/mess a bit with them
- use two separate exports for images and images/dev (always in OMV) with the exact options found in FOG (so something almost identical to the setup of the FOG server)
The storage configuration is the same i posted before: screenshot, where 10.21.100.29 is the OMV server
-
@giolnl said:
the FOG server looks like yours, some things I already tried:
You need an NFS Export for both the /images directory and the /images/dev directories - wherever those may be. For troubleshooting purposes, set world-writable permissions (777) on the /images directory just to see it get going without unnecessary complications.
-
@giolnl I guess your /etc/exports on OMV should look like this:
/export/backup 10.21.0.0/16(rw,subtree_check,secure) /export/images (rw,subtree_check,secure) /export/images/dev (rw,subtree_check,secure) # NFSv4 - pseudo filesystem root /export 10.21.0.0/16(ro,fsid=0,root_squash,no_subtree_check,hide) /export (ro,fsid=0,root_squash,no_subtree_check,hide)Maybe even add the NFS parameters originally used by FOG (see ch3i’s post).
Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)
Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG
-
@Uncle-Frank I’d highly recommend that.
-
@Uncle-Frank
I’ve just tested it with no successEDIT: I noticed a new thing, all the other tests I’ve done I’ve always set the task from the web GUI and then start the client with PXE, this time I just started the client without any task pending and then tried to do a quick image and now I can see this error in the Image list in the GUI under “image size: ON SERVER”, don’t know if it can be usefull or not:
FOGFTP: Failed to connect. Host: 10.21.100.29, Error: Undefined property: FOGFTP::$linkEND EDIT
/etc/exports on OMV:
# /etc/exports: the access control list for filesystems which may be exported # to NFS clients. See exports(5). /export/backup 10.21.0.0/16(rw,subtree_check,secure) /export/images (rw,subtree_check,secure) /export/dev (rw,subtree_check,secure) # NFSv4 - pseudo filesystem root /export 10.21.0.0/16(ro,fsid=0,root_squash,no_subtree_check,hide) /export (ro,fsid=0,root_squash,no_subtree_check,hide)Same errors:
Sep 30 05:48:43 omv rpc.mountd[18040]: refused mount request from 10.21.1.20 for /images/ (/): not exportedand permission denied on the client
root@omv:~# ll -aR /export/images/ /export/images/: total 12 drwxrwxrwx+ 3 root users 32 Jul 30 17:10 . drwxr-xr-x 5 root root 4096 Sep 30 08:35 .. -rwxrwxrwx+ 1 nobody users 0 Jul 30 17:10 .mntcheck drwxrwsrwx+ 2 nobody users 22 Jul 30 17:10 dev /export/images/dev: total 8 drwxrwsrwx+ 2 nobody users 22 Jul 30 17:10 . drwxrwxrwx+ 3 root users 32 Jul 30 17:10 .. -rwxrwxrwx+ 1 nobody users 0 Jul 30 17:10 .mntcheck -
/export/images/devnot/export/devI reckon -
# /export/backup 10.21.0.0/16(rw,subtree_check,secure) #Commented out the above line just for troubleshooting purposes. /export/images *(ro,sync,no_wdelay,no_subtree_check,insecure_locks,no_root_squash,insecure,fsid=0) /export/images/dev *(rw,async,no_wdelay,no_subtree_check,no_root_squash,insecure,fsid=1) -
@Uncle-Frank
The fact is that OMV doesn’t le me export “directly” a subdirectory, it let me create a shared folder which point to /export/images/dev/ but then it exports it as /export/dev/ which is linked somehow to /export/images/dev/ (I don’t know how to be honest),
Anyway I made some tests with a “forced” configuration (meaning I actually bypassed the GUI of OMV) and everything worked, so the problem is in the OMV configuration, I’m still doing some tests but i think the problem is related to some ACL you can set on those shared folders and theirs subdirectories.I’ll be back when I’ll be sure to have a working “minimal” configuraton for both FOG and OMV.
Thanks anyone for pointing me to the right direction!
-
Guess I was wrong about the ACL, in the end what made the difference was the fact that during the “forced” tests I used “/export/images” as image path in the storage configuration instead of “/images” (because of the missing “/export” lines in /etc/exports)
Here some screenshots with the working configuration I’m actually using:
OMV Shared folder configuration
OMV Shared folder ACL configuration
OMV NFS shares
FOG Storage configuration
The actual /etc/exports file on OMV:
# /etc/exports: the access control list for filesystems which may be exported # to NFS clients. See exports(5). /export/images (rw,subtree_check,secure) /export/backup (rw,subtree_check,secure) # NFSv4 - pseudo filesystem root /export (ro,fsid=0,root_squash,no_subtree_check,hide)Could be that NSFv4 pseudo fs to cause the problem? Maybe by giving FOG the full path I somehow bypass it, just a random guess tho, I don’t know how NFS works apart from the basics.
-
With /export/images/dev missing in /etc/exports is it working now??
