Could not mount images folder (bin/fog.download) ... failed: Operation not supported
-
Hi folks been a while but we’re back using FOG again I’ve setup a multiple server/location setup all running on Ubuntu 18.04 and FOG 1.55. with a management node and storage node in our main site both set as Master, with the Storage node being Master for all bar one of 11 remote storage nodes.
Using location, WOL broadcast and LDAP plugins.
We had been deploying and capturing images fine after initial setup this week but today started getting this error.
Nothing has changed far as I can see and I’m not in the office today to test the machines myself but will look to do a debug boot tomorrow to see if i can see any errors,
I’ve found similar reports in here with error “permission denied” but not for Operation not supported.
My server setup (same across storage and normal nodes) is:
Ubuntu 18.04.3 virtual machines
2 drives, (dev/sda) x 30GB for OS, (dev/sdb) x 60 GB for images
/dev/sdb mounted to /mnt/images with fstab entry
/images directory has .mntcheck in required locations
ownership of the directory was set to fogproject user during setup
images appear to have owner set to root but changing this to fogproject user does not resolve issue.This is happening when deploying both pre and post sysprepped images. We have another error on the post-syprepped ones generating an error “Windows could not complete the installation” but this is down to the fogclient being enabled on the base image when sysprep taken I think. I can sort that separately.
Any suggestions on above would be great.
And while I’m here can I just say now great the UI is looking these days! i had already seen the improvements the team had put in place when i was last using FOG about 3 odd years ago but its looking really slick now.
I’ve fully updated my documentation on the installation of above for our service desk teams and in case thats of use to anyone here I’m happy to share.
regards Tom
-
What does the output of this command look like when run on your fog server?
sudo showmount -e 127.0.0.1
-
@george1421 said in Could not mount images folder (bin/fog.download) ... failed: Operation not supported:
sudo showmount -e 127.0.0.1
Hi George,
Output of command as follows
clnt_create: RPC: Program not registered
I’ve checked NFS server service:
systemctl status nfs-server
and was not started showing following output:
● nfs-server.service - NFS server and services Loaded: loaded (/lib/systemd/system/nfs-server.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Tue 2020-02-04 21:58:25 UTC; 21h ago Process: 905 ExecStopPost=/usr/sbin/exportfs -f (code=exited, status=0/SUCCESS) Process: 903 ExecStopPost=/usr/sbin/exportfs -au (code=exited, status=0/SUCCESS) Process: 891 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=1/FAILURE) Feb 04 21:58:25 geo01vfog01 systemd[1]: Starting NFS server and services... Feb 04 21:58:25 geo01vfog01 exportfs[891]: exportfs: Failed to stat /mnt/images/dev: No such file or directory Feb 04 21:58:25 geo01vfog01 systemd[1]: nfs-server.service: Control process exited, code=exited status=1 Feb 04 21:58:25 geo01vfog01 systemd[1]: nfs-server.service: Failed with result 'exit-code'. Feb 04 21:58:25 geo01vfog01 systemd[1]: Stopped NFS server and services.
I was able to start it again via
sudo systemctl start nfs-server
This was the same on both the Master (normal) and Master (storage) nodes.
I’ve enabled debugging to see if I can figure out why NFS server stopped which I’m guessing is the cause of the issue I’m seeing?
sudo vi /etc/default/nfs-kernel-server
Added following and restarted NFS service
RPCMOUNTDOPTS="--manage-gids --debug all"
Also seeing loads of these errors in syslog not sure if related?
Feb 5 19:05:06 fogserver systemd-resolved[842]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-20 18-0001, retrying transaction with reduced feature level UDP. Feb 5 19:05:19 geo01vfog01 in.tftpd[21004]: tftp: client does not accept options Feb 5 19:05:49 geo01vfog01 in.tftpd[21104]: tftp: client does not accept options Feb 5 19:05:52 geo01vfog01 in.tftpd[21114]: tftp: client does not accept options Feb 5 19:06:45 geo01vfog01 in.tftpd[21284]: tftp: client does not accept options Feb 5 19:06:56 geo01vfog01 in.tftpd[21319]: tftp: client does not accept options
I’ll be able to confirm tomorrow when I’m back in the office that this is resolved or not.
I’m seeing a separate issue with the locations plugin doing something odd but will log separate ticket for that.
Thanks again George
regards Tom
-
@Kiweegie The expected output from the show mount is a list of nfs shares. The rpc program not registered is a bit unexpected.
What is the output of
cat /etc/exports
-
@Kiweegie said in Could not mount images folder (bin/fog.download) ... failed: Operation not supported:
… Failed to stat /mnt/images/dev: No such file or directory
Please run
ls -alR /mnt/images/
and post output here. -
Hi @george1421
output as follows on all nodes.
/mnt//images *(ro,sync,no_wdelay,no_subtree_check,insecure_locks,no_root_squash,insecure,fsid=0) /mnt/images/dev *(rw,async,no_wdelay,no_subtree_check,no_root_squash,insecure,fsid=1)
cheers Tom
-
@Sebastian-Roth said in Could not mount images folder (bin/fog.download) ... failed: Operation not supported:
ls -alR /mnt/images/
Here you go Sebastien.
Output from geovfog01 (Georgia normal master server)
admin@geo01vfog01:~$ sudo ls -alR /mnt/images/ [sudo] password for admin: /mnt/images/: total 52 drwxr-xr-x 10 fogproject root 4096 Feb 4 18:52 . drwxr-xr-x 3 root root 4096 Jan 30 11:25 .. drwxrwxrwx 3 fogproject root 4096 Feb 4 18:52 dev drwx------ 2 fogproject root 16384 Jan 30 11:25 lost+found -rwxrwxrwx 1 fogproject root 0 Jan 30 14:58 .mntcheck drwxrwxrwx 2 fogproject root 4096 Jan 30 13:30 postdownloadscripts drwxrwxrwx 2 fogproject root 4096 Feb 4 18:18 Toysrus_Post_Sysprep drwxrwxrwx 2 fogproject root 4096 Feb 4 18:03 Toysrus_Pre_Sysprep drwxrwxrwx 2 fogproject root 4096 Feb 5 15:40 Mattel_Post_Sysprep drwxrwxrwx 2 fogproject root 4096 Feb 4 18:39 Mattel_Presysprep drwxrwxrwx 2 fogproject root 4096 Feb 4 15:14 Windows10_presysprep /mnt/images/dev: total 12 drwxrwxrwx 3 fogproject root 4096 Feb 4 18:52 . drwxr-xr-x 10 fogproject root 4096 Feb 4 18:52 .. -rwxrwxrwx 1 fogproject root 0 Jan 30 13:30 .mntcheck drwxrwxrwx 2 fogproject root 4096 Jan 30 13:30 postinitscripts /mnt/images/dev/postinitscripts: total 12 drwxrwxrwx 2 fogproject root 4096 Jan 30 13:30 . drwxrwxrwx 3 fogproject root 4096 Feb 4 18:52 .. -rwxrwxrwx 1 fogproject root 249 Jan 30 13:30 fog.postinit /mnt/images/lost+found: total 20 drwx------ 2 fogproject root 16384 Jan 30 11:25 . drwxr-xr-x 10 fogproject root 4096 Feb 4 18:52 .. /mnt/images/postdownloadscripts: total 12 drwxrwxrwx 2 fogproject root 4096 Jan 30 13:30 . drwxr-xr-x 10 fogproject root 4096 Feb 4 18:52 .. -rwxrwxrwx 1 fogproject root 235 Jan 30 13:30 fog.postdownload /mnt/images/Toysrus_Post_Sysprep: total 7629888 drwxrwxrwx 2 fogproject root 4096 Feb 4 18:18 . drwxr-xr-x 10 fogproject root 4096 Feb 4 18:52 .. -rwxrwxrwx 1 fogproject root 5 Feb 4 18:14 d1.fixed_size_partitions -rwxrwxrwx 1 fogproject root 1048576 Feb 4 18:14 d1.mbr -rwxrwxrwx 1 fogproject root 190 Feb 4 18:14 d1.minimum.partitions -rwxrwxrwx 1 fogproject root 15 Feb 4 18:14 d1.original.fstypes -rwxrwxrwx 1 fogproject root 0 Feb 4 18:14 d1.original.swapuuids -rwxrwxrwx 1 fogproject root 9265465 Feb 4 18:14 d1p1.img -rwxrwxrwx 1 fogproject root 7802656523 Feb 4 18:18 d1p2.img -rwxrwxrwx 1 fogproject root 190 Feb 4 18:14 d1.partitions /mnt/images/Toysrus_Pre_Sysprep: total 8060100 drwxrwxrwx 2 fogproject root 4096 Feb 4 18:03 . drwxr-xr-x 10 fogproject root 4096 Feb 4 18:52 .. -rwxrwxrwx 1 fogproject root 5 Feb 4 17:59 d1.fixed_size_partitions -rwxrwxrwx 1 fogproject root 1048576 Feb 4 17:59 d1.mbr -rwxrwxrwx 1 fogproject root 190 Feb 4 17:59 d1.minimum.partitions -rwxrwxrwx 1 fogproject root 15 Feb 4 17:59 d1.original.fstypes -rwxrwxrwx 1 fogproject root 0 Feb 4 17:59 d1.original.swapuuids -rwxrwxrwx 1 fogproject root 453635778 Feb 4 17:59 d1p1.img -rwxrwxrwx 1 fogproject root 7798828240 Feb 4 18:03 d1p2.img -rwxrwxrwx 1 fogproject root 190 Feb 4 17:59 d1.partitions /mnt/images/Mattel_Post_Sysprep: total 7215252 drwxrwxrwx 2 fogproject root 4096 Feb 5 15:40 . drwxr-xr-x 10 fogproject root 4096 Feb 4 18:52 .. -rwxrwxrwx 1 fogproject root 5 Feb 4 18:46 d1.fixed_size_partitions -rwxr-xr-x 1 fogproject root 1048576 Feb 5 15:40 d1.mbr -rwxr-xr-x 1 fogproject root 190 Feb 5 15:40 d1.minimum.partitions -rwxrwxrwx 1 fogproject root 15 Feb 4 18:46 d1.original.fstypes -rwxrwxrwx 1 fogproject root 0 Feb 4 18:46 d1.original.swapuuids -rwxr-xr-x 1 fogproject root 10387064 Feb 5 15:40 d1p1.img -rwxr-xr-x 1 fogproject root 7376949941 Feb 5 15:42 d1p2.img -rwxr-xr-x 1 fogproject root 190 Feb 5 15:40 d1.partitions /mnt/images/Mattel_Presysprep: total 7087908 drwxrwxrwx 2 fogproject root 4096 Feb 4 18:39 . drwxr-xr-x 10 fogproject root 4096 Feb 4 18:52 .. -rwxrwxrwx 1 fogproject root 5 Feb 4 18:35 d1.fixed_size_partitions -rwxrwxrwx 1 fogproject root 1048576 Feb 4 18:35 d1.mbr -rwxrwxrwx 1 fogproject root 190 Feb 4 18:35 d1.minimum.partitions -rwxrwxrwx 1 fogproject root 15 Feb 4 18:35 d1.original.fstypes -rwxrwxrwx 1 fogproject root 0 Feb 4 18:35 d1.original.swapuuids -rwxrwxrwx 1 fogproject root 454330540 Feb 4 18:35 d1p1.img -rwxrwxrwx 1 fogproject root 6802601965 Feb 4 18:39 d1p2.img -rwxrwxrwx 1 fogproject root 190 Feb 4 18:35 d1.partitions /mnt/images/Windows10_presysprep: total 8864012 drwxrwxrwx 2 fogproject root 4096 Feb 4 15:14 . drwxr-xr-x 10 fogproject root 4096 Feb 4 18:52 .. -rwxrwxrwx 1 fogproject root 13 Feb 4 15:09 d1.fixed_size_partitions -rwxrwxrwx 1 fogproject root 1048576 Feb 4 15:09 d1.mbr -rwxrwxrwx 1 fogproject root 861 Feb 4 15:09 d1.minimum.partitions -rwxrwxrwx 1 fogproject root 15 Feb 4 15:09 d1.original.fstypes -rwxrwxrwx 1 fogproject root 0 Feb 4 15:09 d1.original.swapuuids -rwxrwxrwx 1 fogproject root 444522502 Feb 4 15:09 d1p1.img -rwxrwxrwx 1 fogproject root 11385185 Feb 4 15:10 d1p2.img -rwxrwxrwx 1 fogproject root 174678 Feb 4 15:10 d1p3.img -rwxrwxrwx 1 fogproject root 8619581073 Feb 4 15:14 d1p4.img -rwxrwxrwx 1 fogproject root 861 Feb 4 15:09 d1.partitions admin@geo01vfog01:~$
Output from geovfog02 (Georgia storage master server)
admin@geo01vfog02:~$ sudo !! sudo ls -alR /mnt/images/ [sudo] password for admin: /mnt/images/: total 36 drwxr-xr-x 6 fogproject root 4096 Feb 4 16:13 . drwxr-xr-x 3 root root 4096 Jan 30 18:20 .. drwxrwxrwx 3 fogproject root 4096 Feb 4 16:13 dev drwx------ 2 fogproject root 16384 Jan 30 18:20 lost+found -rwxrwxrwx 1 fogproject root 0 Jan 30 19:19 .mntcheck drwxrwxrwx 2 fogproject root 4096 Jan 30 19:19 postdownloadscripts drwxrwxrwx 2 fogproject root 4096 Feb 4 16:13 Mattel_Post_Sysprep /mnt/images/dev: total 12 drwxrwxrwx 3 fogproject root 4096 Feb 4 16:13 . drwxr-xr-x 6 fogproject root 4096 Feb 4 16:13 .. -rwxrwxrwx 1 fogproject root 0 Jan 30 19:19 .mntcheck drwxrwxrwx 2 fogproject root 4096 Jan 30 19:19 postinitscripts /mnt/images/dev/postinitscripts: total 12 drwxrwxrwx 2 fogproject root 4096 Jan 30 19:19 . drwxrwxrwx 3 fogproject root 4096 Feb 4 16:13 .. -rwxrwxrwx 1 fogproject root 249 Jan 30 19:19 fog.postinit /mnt/images/lost+found: total 20 drwx------ 2 fogproject root 16384 Jan 30 18:20 . drwxr-xr-x 6 fogproject root 4096 Feb 4 16:13 .. /mnt/images/postdownloadscripts: total 12 drwxrwxrwx 2 fogproject root 4096 Jan 30 19:19 . drwxr-xr-x 6 fogproject root 4096 Feb 4 16:13 .. -rwxrwxrwx 1 fogproject root 235 Jan 30 19:19 fog.postdownload /mnt/images/Mattel_Post_Sysprep: total 7215252 drwxrwxrwx 2 fogproject root 4096 Feb 4 16:13 . drwxr-xr-x 6 fogproject root 4096 Feb 4 16:13 .. -rwxrwxrwx 1 fogproject root 5 Feb 4 16:09 d1.fixed_size_partitions -rwxrwxrwx 1 fogproject root 1048576 Feb 4 16:09 d1.mbr -rwxrwxrwx 1 fogproject root 190 Feb 4 16:09 d1.minimum.partitions -rwxrwxrwx 1 fogproject root 15 Feb 4 16:09 d1.original.fstypes -rwxrwxrwx 1 fogproject root 0 Feb 4 16:09 d1.original.swapuuids -rwxrwxrwx 1 fogproject root 10387064 Feb 4 16:09 d1p1.img -rwxrwxrwx 1 fogproject root 7376949941 Feb 4 16:13 d1p2.img -rwxrwxrwx 1 fogproject root 190 Feb 4 16:09 d1.partitions admin@gla01vfog02:~$ sudo ls -alR /mnt/images/ /mnt/images/: total 36 drwxr-xr-x 6 fogproject root 4096 Feb 4 16:13 . drwxr-xr-x 3 root root 4096 Jan 30 18:20 .. drwxrwxrwx 3 fogproject root 4096 Feb 4 16:13 dev drwx------ 2 fogproject root 16384 Jan 30 18:20 lost+found -rwxrwxrwx 1 fogproject root 0 Jan 30 19:19 .mntcheck drwxrwxrwx 2 fogproject root 4096 Jan 30 19:19 postdownloadscripts drwxrwxrwx 2 fogproject root 4096 Feb 4 16:13 Mattel_Post_Sysprep /mnt/images/dev: total 12 drwxrwxrwx 3 fogproject root 4096 Feb 4 16:13 . drwxr-xr-x 6 fogproject root 4096 Feb 4 16:13 .. -rwxrwxrwx 1 fogproject root 0 Jan 30 19:19 .mntcheck drwxrwxrwx 2 fogproject root 4096 Jan 30 19:19 postinitscripts /mnt/images/dev/postinitscripts: total 12 drwxrwxrwx 2 fogproject root 4096 Jan 30 19:19 . drwxrwxrwx 3 fogproject root 4096 Feb 4 16:13 .. -rwxrwxrwx 1 fogproject root 249 Jan 30 19:19 fog.postinit /mnt/images/lost+found: total 20 drwx------ 2 fogproject root 16384 Jan 30 18:20 . drwxr-xr-x 6 fogproject root 4096 Feb 4 16:13 .. /mnt/images/postdownloadscripts: total 12 drwxrwxrwx 2 fogproject root 4096 Jan 30 19:19 . drwxr-xr-x 6 fogproject root 4096 Feb 4 16:13 .. -rwxrwxrwx 1 fogproject root 235 Jan 30 19:19 fog.postdownload /mnt/images/Mattel_Post_Sysprep: total 7215252 drwxrwxrwx 2 fogproject root 4096 Feb 4 16:13 . drwxr-xr-x 6 fogproject root 4096 Feb 4 16:13 .. -rwxrwxrwx 1 fogproject root 5 Feb 4 16:09 d1.fixed_size_partitions -rwxrwxrwx 1 fogproject root 1048576 Feb 4 16:09 d1.mbr -rwxrwxrwx 1 fogproject root 190 Feb 4 16:09 d1.minimum.partitions -rwxrwxrwx 1 fogproject root 15 Feb 4 16:09 d1.original.fstypes -rwxrwxrwx 1 fogproject root 0 Feb 4 16:09 d1.original.swapuuids -rwxrwxrwx 1 fogproject root 10387064 Feb 4 16:09 d1p1.img -rwxrwxrwx 1 fogproject root 7376949941 Feb 4 16:13 d1p2.img -rwxrwxrwx 1 fogproject root 190 Feb 4 16:09 d1.partitions admin@geo01vfog02:~$
I’ve added the server setup to another ticket ref location plugin but might be of use to see here also.
-
@Kiweegie said in Could not mount images folder (bin/fog.download) ... failed: Operation not supported:
/mnt//images *(ro, …
While I don’t think this can cause the issue I’d still change that to
/mnt/images
.Beside that I see different IP addresses:
10.166.141.199
in the picture of your initial post while that IP is not in the list you posted just now… -
@Kiweegie ok your exports file is correct (hopefully the double slash in the images share was a type-o. That double slash // would keep the nfs server from starting). Lets make sure we have this right. You are NOT using or storing any images in /images directory path only the /mnt/images and /mnt/images is mounted on a local hard drive on the fog server? If you are using only /mnt/images you will need to update the storage node configuration for this server and change the ftp and nfs paths.
If that was/is the case I might go about it a bit differently just to keep everything clean.
Edit: Well it would help if I would read the entire thread before responding to avoid duplicating what Sebastian posted.
-
@Sebastian-Roth good spot on the // it was there in exports on geovfog02 server. I’ve now corrected that I’ll blame my poor eyesight on that one…
I’ve double checked all other nodes to be on the safe side and it was only this one.
Ignore the ip from original image was screenshot from an earlier build, albeit same error.
edit: I’ve corrected the original image to avoid confusion and also provide better image - notepad++ with Deep Black theme appplied.
regards Tom
-
Hi both not seeing this error any more so can be marked as resolved. I was not seeing the issue before editing the // in the exports file so can’t say that was the fix. Not sure what has changed if anything here other than permissions on the images themselves within the /mnt/images directory.
Ownership was set on the folder to fogproject:root but the images are getting set as root:root. I amended this to fogproject:root again so not sure if that is needed? If so can add sticky bit to get around this perhaps.
EDIT: Ignore above comment ref permissions - some are set as fogproject:root and some as root:root - both work so the permissions is a red herring.
Thoughts welcome
regards Tom
-
@Kiweegie said in Could not mount images folder (bin/fog.download) ... failed: Operation not supported:
Thoughts welcome
Do you have firewall restrictions in your MPLS or the VPN tunnel?