Error mounting filesystem /images/dev on an NFS
-
Hi all, I’ve been going slightly crazy trying to figure this out. I have a Cisco NSS nas unit which I’ve created a share on for the FOG server.
I’ve followed the guide here ([url]http://www.fogproject.org/wiki/index.php?title=FOGUserGuide#Separate_NFS_Server[/url]) and performed the following commands:sudo mv /images /imagesold
sudo mkdir /images
sudo nano /etc/fstaband added the line
nasipaddress:/fog /images nfs defaults 0 0I’ve also created a dev folder, and done sudo touch .mntcheck
Both the images folder and dev folder are chown fog:root and the permissions are 777 on all files and folders within. From the shell, I can navigate into the dev folder, create folders and delete them.
When I try to take an image of the machine I am working on, I receive
“Fatal Error: Failed to mount NFS volume”Above it, I see "/images/dev failed, reason given by server: Permission Denied
mount: mounting :/images/dev on /images failed: Bad File DescriptorHowever if I manually try and mount the dev folder somewhere in the OS, I can do what I want in there, create, delete etc.
OS Is Ubuntu 12.04
Can anyone offer any suggestions at all?
-
The only comment I can make at the moment is that you are not alone in this problem.
[url]http://fogproject.org/forum/threads/fog-server-fatal-error-failed-to-mount-nfs-volume.115/[/url]
I’m about to wipe my systems and start from scratch to see if I can figure this out today.
-
Thanks. I’m hoping not to wipe, and certainly hoping to not give up a 10TB NAS. Once you’re done I’d be happy to compare configs. It seems very strange, but almost certainly a fog issue rather than an OS issue since it all mounts very happily and cleanly in the OS…
-
Ok let’s add a little information here to see if this helps anyone.
Here’s an ls -la of the /images folder
[CODE]8 drwxrwxrwx+ 2 root root 4096 Apr 8 17:49 dev
-rwxrwxrwx+ 1 root root 0 Apr 8 17:15 .mntcheck[/CODE]
And of the images/dev folder
[CODE]-rwxrwxrwx+ 1 root root 0 Apr 8 17:49 .mntcheck[/CODE]Here’s the output of ps ax|grep nfs
[CODE]
523 ? S< 0:00 [nfsiod]
1307 ? S< 0:00 [nfsd4]
1308 ? S< 0:00 [nfsd4_callbacks]
1309 ? S 0:00 [nfsd]
1310 ? S 0:00 [nfsd]
1311 ? S 0:00 [nfsd]
1312 ? S 0:00 [nfsd]
1313 ? S 0:00 [nfsd]
1314 ? S 0:00 [nfsd]
1315 ? S 0:00 [nfsd]
1316 ? S 0:00 [nfsd]
5885 pts/0 S+ 0:00 grep --color=auto nfs
[/CODE]
Here’s what’s in /etc/exports
[CODE]
/images *(ro,sync,no_wdelay,insecure_locks,no_root_squash,insecure)
/images/dev *(rw,sync,no_wdelay,no_root_squash,insecure)
[/CODE]
ExportFS
[CODE]
/images <world>
/images/dev <world>
[/CODE]
And finally the /etc/fstab
[CODE]
IPADDRESS:/fog /images nfs defaults 0 0
[/CODE]
Hopefully this will be of benefit to someone!
32bit Ubuntu 12.04 LTS by the way.Edit:
Also decided to run the exportfs -a command as per this guide [url]http://www.fogproject.org/wiki/index.php?title=Change_NFS_location[/url]
This was the output which is… interesting to me
[CODE]
exportfs: /etc/exports [1]: Neither ‘subtree_check’ or ‘no_subtree_check’ specified for export “*:/images”.
Assuming default behaviour (‘no_subtree_check’).
NOTE: this default has changed since nfs-utils version 1.0.xexportfs: /etc/exports [2]: Neither ‘subtree_check’ or ‘no_subtree_check’ specified for export “*:/images/dev”.
Assuming default behaviour (‘no_subtree_check’).
NOTE: this default has changed since nfs-utils version 1.0.xexportfs: /images/dev does not support NFS export
exportfs: /images does not support NFS export
[/CODE]Second Edit -
Final update. Nope I am still not working here. I went ahead and mounted the device to the images folder as cifs share instead of an nfs share. Again, full permissions to do anything I want on there, navigated to the folder, created files, deleted others to confirm access.Booted off the NIC. STILL got permission denied.
Checked the NAS log files, not even a hint of it touching the NAS at all.I am tearing my hair out on this too. And it might be something really obvious. One thing I can absolutely confirm is that it’ll work on the local drive, so it’s just something to do with the NAS.
I’ve gone and recreated the entire array on the NAS but still no joy!
Third Edit -
(I’m doing it like this so as not to bump threads unnecessarily)I went and monitored the /dev/syslog and found this
[CODE]
Apr 9 15:56:46 SYSNAME rpc.mountd[1326]: authenticated mount request from IPADDRESS:834 for /images/dev (/images/dev)
Apr 9 15:56:46 SYSNAME rpc.mountd[1326]: qword_eol: fflush failed: errno 22 (Invalid argument)
Apr 9 15:56:46 SYSNAME rpc.mountd[1326]: Cannot export /images/dev, possibly unsupported filesystem or fsid= required
[/CODE]
What’s interesting is the filesystem is EXT4, but I had the same issue on EXT3. So I’m now reformatting my NAS to EXT3 and will update again when this is done -
I feel like the issue is the client not having read/write permissions on the NAS. Like you said, the FOG system has permissions, and the mount is fine. I am fairly sure that it is not a problem with the NAS, as (mine at least) has no password required, and guest access to read and write enabled, and the export has No_root_squash which should allow non-root to have full permissions.
I have a feeling that there is a password field or setting in FOG that is being over looked.
I also didn’t get a chance to build from scratch today. I don’t want to take down the FOG system I have because it is in production and the system I was going to use has a hardware failure. When I get my parts in, I’ll try again, using these directions:
[url]http://fogproject.org/forum/threads/request-fog-nas-tutorial.419/[/url]
For your /images and /images/dev folder it looks like you have the ownership of root, when it should be (according to the above thread) fog.
-
Thanks for the feedback. I’ll change the ownership to fog:root and give it a go.
I too am having thoughts that it’s the TFTP client that doesn’t have permission to mount rather than fog. I feel like I should be giving the TFTP client a username or password for the NFS share but can’t seem to find a setting for it.
Edit:
Ok tried the fog:root permissions. No luck
Tried following that guide to mount the NAS to another directory. Nothing. Actually get failed to connect on most things.Here’s what I did.
Created the directory /mnt/pools/A/A0/fog
Chowned the directory recursively as fog:root and chmodded it as 777 as wellMy /etc/fstab looks like this
[CODE]
IPADDRESS:/fog/ /mnt/pools/A/A0/fog nfs user,bg,noexec 0 0
[/CODE]It mounts, again I can navigate to it and do what I want in it, but the FOG dashboard says failed to connect if I try and create it. Unlike that guide, It doesn’t show up for me under disk information on either the DefaultMember group or it’s own connection.
Hmm.
Edit 2:
Just to be sure I changed the /etc/fstab to mount it to /mnt/pools/A/A0/images rather than /fog
Still no bueno! -
/mnt/pools/A/A0/images is the directory on his NAS. You should still mount it to /images.
To be clear, if you have the drive mounted correctly without any other changes, to /images , the FOG Dashboard should show the correct disk information. I really do believe that the mount is correct, and the problem is with the client not having correct permissions.
-
SO, it works!
I followed these instructions: [url]http://fogproject.org/forum/threads/request-fog-nas-tutorial.419/#post-11178[/url]
Instead of mnt/pools/A/A0/images, you will use your mount, and of course, your ip to your NAS instead of the one he used. The password should be fog/password unless you have changed the FOG password (not the system password).
A word of caution, setting the node as Master (which I believe is needed for this to work) will overwrite your other nodes if you have any. Backup any images you need to keep.
-
Alright confirmed I can get it further than before. Thank you!
New issue however. I now get
[CODE]
Mounting File System…DONE
Checking Mounted File System…AN ERROR HAS BEEN DETECTED
Fatal Error: Failed to mount NFS volume
[/CODE]Now that’s wierd
-
Is your mount correct (can you still access it in the OS)? Your NAS must be mounted to the /images directory on the fog system.
Are the settings in your storage node correct for your NAS?
-
Ok, so I’ve gone ahead and unmounted the drives, removed from my fstab, removed any tasks outstanding and rebooted to make sure it’s all totally and utterly clear.
I’ve moved the originally installed /images folder back to /images (all I did before was move it from /images to /imagesold
I’ve made a directory at /media/images where I can mount my share. It’s chowned to fog:root, and chmodded to 777
fstab now looks like this
[CODE]
IPADDRESS:/fog /media/images nfs defaults, 0 0
[/CODE]Then, sudo mount -a to mount the drive
Ok, back into the fog dashboard
Created storage nodeIP address is the IP address of the NAS
image location /fog/
Username and Password is the same username and password I use to login to the fog dashboard
Click updateCreate new upload task annnnnddd
same error as before, mounted - done. Checking mounted file system - error, failed to mountSo, with the drive mounted
sudo chmod -R 755 /images
sudo chmod -R 777 /images/dev
sudo chown -R fog:root /imagesTry again and nope.
So, chmod -R 777 /imagesNope!
Head = Brickwall
-
UPDATE! = Success!
I have it working I deleted and recreated the .mntcheck files in both the directories and BOOM suddenly it started working properly.
I’m guessing that I must have corrupted the .mntcheck files with all the attempts
UGH three days!
I’m going for lunch and I’ll update exactly what my config is when I get back. Hopefully someone who searches in the future will be able to benefit from the information.
-
Alrighty so the test has worked and I’ve got the FTP error issue but it looks like it’s uploading ok.
I’m running another test because suddenly in the dashboard, I can’t see anything relating to the size again on the DefaultMaster, whereas I could before.
I will report back
-
Is it uploading but not renaming and moving the img files from /dev?
-
[quote=“mootsfox, post: 11569, member: 1747”]Is it uploading but not renaming and moving the img files from /dev?[/quote]
It was but I changed the TFTP password on the NAS and it looks like it went through. Right now it’s trying to upload an image for me and has 30 minutes left. I have some very strange things going on right now. Nothing in the Bandwidth - Receive graph. In fact right now it shows the time as 7pm, rather than 2pm. However if I manually go to the dev folder on the NAS and do an ls -lh I can see the file growing. So it’s not all bad I guess. Maybe I’ll map the NAS to my /images folder since that was able to see the disk size quite happily -
Got it! It’s working!
I’m going to do a fresh install of this server I think. The time is messed up on the bandwidth receive graph.Ok so for future individuals, here’s my little guide during installation
Adding a NFS share
First, get rid of the old /images directory. Move it somewhere safe
[B]sudo mv /images /imagesold[/B]
[B]sudo mkdir /images[/B]
[B]sudo chown –R fog:root /images[/B]
[B]sudo chmod –R 777 /images[/B]Next, let’s edit your fstab
[B]sudo nano /etc/fstab[/B]Add the line in your fstab for the nfs drive
[B]IPADDRESS:/sharename /images nfs defaults 0 0[/B]Mount the drive
[B]sudo mount -a[/B]Then reapply the permissions to the folder now that the drive is mounted. Create the dev folder and .mntcheck folders
[B]sudo mkdir /images/dev[/B]
[B]sudo touch /images/dev/.mntcheck[/B]
[B]sudo touch /images/.mntcheck[/B]
[FONT=arial][SIZE=3][B]sudo chown –R fog:root /images[/B][/SIZE][/FONT]
[SIZE=3][FONT=arial][B]sudo chmod –R 755 /images[/B][/FONT][/SIZE]
[SIZE=3][FONT=arial][B]sudo chmod –R 777 /images/dev[/B][/FONT][/SIZE]Then login to the FOG panel, go to storage management, click Add Storage Node and then use these values
Storage Node Name: Whatever you want
Storage Node Description: Whatever you want
IP Address: IP Address of your NFS share. JUST THE IP ADDRESS
Max Clients: Whatever you want, I generally stick 10 in there
Is Master Node: Yes. Note - if you tick this, it’ll wipe images you’ve already created
Storage group: Default
Image location: Enter the folder on your NAS drive that your images are stored in
Is enabled: Tick it
Management Username: the username you use to login to the FOG control panel
Management Password: the password you use to login to the FOG control panelClick update, and you’re done!
-
Followed your directions and am currently uploading an image to test it.
Why dd you add a storage node instead of editing the DefaultMember?
-
Just so the graph would work for remaining space.
-
My computer finished, and I had the “FTP connection…” error again. While the system was running, I changed the DefaultMember back to defaults, and created a new one, and the system finished, and the files moved as they should have.
So I have deleted that image and am trying it again.
-
How did you get on?
My test worked ok, but the bandwidth graph still is 7 hours ahead and refuses to show information. Fortunately the system appears to actually be working!