Permission Denied when trying to image after update to 1.1.1/1.1.2



  • Yesterday I updated my Fog server to 1.1.1. When trying to image, I now get the following: [attached file]

    I have done the following:

    chmod 777 -R /images
    chmod 777 -R /images/dev
    touch /images/.mntcheck
    touch /images/dev/.mntcheck
    touch /fog/images/.mntcheck
    chmod 777 -R /fog/images

    I’ve changed the TFTP password in the GUI to match the password in the config.class.php file.
    I’ve done the same with the FTP password.

    I’m perplexed, and just frustrated at this point. Any ideas, please?

    [url="/_imported_xf_attachments/1/1068_Capture.PNG?:"]Capture.PNG[/url]



  • Thanks, guys. I finally got it working. It was a mess…I must have just started changing things willy-nilly and stopped paying attention to what I was doing.
    The export file, as ianabc stated was incorrect. I did:
    nano /etc/exports
    [SIZE=2]pointed the first line to [/SIZE]fog/images *(ro,sync,no_wdelay,insecure_locks,no_root_sq$
    [SIZE=2]and my machines immediately started imaging.[/SIZE]

    The funny thing is that yesterday I’m almost certain I had a line in that file pointing to that path. Maybe I didn’t comment out the other one.

    Either way, it appears to be working. Hopefully I didn’t change something else somewhere that’s going to drive me nuts!

    Thanks!!!


  • Testers

    Hi Jamie, I just noticed in your syslog
    [CODE]
    refused mount request from 192.168.3.33 for /fog/images (/): not exported
    [/CODE]
    The fog clients are trying to mount the wrong location, that is almost certainly the problem. Check the image path in your storage configuration, you would either need to update that or change your export list, once they match your NFS problem should go away (and probably the FTP one as well).


  • Senior Developer

    What are the permissions of /images?

    Can you try these steps:
    [code]touch /images/{,dev/}.mntcheck
    chmod -R 777 /images
    service nfs-kernel-server restart[/code]


  • Testers

    [CODE]
    Jun 26 13:22:56 troi rpc.mountd[1691]: authenticated mount request from 192.168.3.33:930 for /images (/images)
    [/CODE]
    Does this correspond with the fog debug task and mount command? did you get any output from that command? Could you turn on debugging for NFS, edit the /etc/default/nfs-kernel-server file to have the line
    [CODE]
    RPCMOUNTDOPTS="–manage-gids --debug all"
    [/CODE]
    Then “service nfs-kernel-server restart” and retry the mount - syslog should get more information this time.


  • Testers

    OK, so it looks like syslog would be the where NFS mounts are logged on your system, is it Ubuntu? I have an ubuntu machine image and I’m deploying it now. Once it is installed I’ll take a look at the NFS setup so we can compare. I’ll post back when it is up and running.

    When I said I thought the network was the problem I meant the networking on the fog server you are running: the installer script runs yum updates and my guess is that a new version of one of these packages broke something.



  • Output of the SYSLOG: (It is refusing the mount request)

    (Attached----it was reporting back as Spam when trying to post)

    [url="/_imported_xf_attachments/1/1078_syslog.txt?:"]syslog.txt[/url]



  • Output of ls -lhrt /var/log:

    drwxr-xr-x 2 root root 4.0K Jun 26 07:03 unattended-upgrades
    -rw-r–r-- 1 root root 235K Jun 26 07:08 dpkg.log
    -rw-r----- 1 mysql adm 0 Jun 26 07:08 mysql.log
    -rw-r----- 1 root adm 0 Jun 26 07:08 apport.log
    drwxr-s— 2 mysql adm 4.0K Jun 26 07:08 mysql
    -rw-r----- 1 syslog adm 1.3M Jun 26 07:08 syslog.1
    drwxr-xr-x 2 root root 4.0K Jun 26 07:08 upstart
    -rw-r----- 1 syslog adm 410K Jun 26 09:30 kern.log
    -rw-r----- 1 syslog adm 5.3K Jun 26 13:22 syslog
    -rw-r----- 1 syslog adm 72K Jun 26 13:23 auth.log
    -rw-rw-r-- 1 root utmp 113K Jun 26 13:23 wtmp
    -rw-rw-r-- 1 root utmp 429K Jun 26 13:23 lastlog



  • [QUOTE]For the disk image sizes, it looks like those are partimage (fog < 1.0) images. This is a known problem, see[URL=‘http://fogproject.org/forum/threads/disk-information-and-image-size-on-server-not-showing-any-data.10953/’] this thread[/URL] for more details on it.[/QUOTE]

    Nope. They have all been made on 1.1.0 over the last few weeks.

    As for the rest being a connection problem—I don’t see how. We haven’t changed a thing on our network. The ONLY thing that has changed is that we updated the Fog server. That’s it. It was upgraded to 1.1.1, and when that didn’t work – 1.1.2.
    It was working just fine on 1.1.0.


  • Testers

    The message [CODE]mount: mounting 192.168.3.8:/fog/images/ on /images failed: Permission denied.[/CODE] is from NFS so it shouldn’t care about FTP or FTP passwords. I still suspect it is a network problem somewhere - your NFS configuration should be able to function with all of the fog services turned off. If you boot one of your clients with a debug task, once it gives you a shell, can you try
    [CODE]
    $ mkdir /images
    $ mount -o nolock,vers=3 IP.OF.YOUR.FOG:/images /images
    [/CODE]
    This should be the NFS mount that your client is doing during an image. When you run that command you should be able to see the server logging the mount, e.g. in /var/log/messages on RHEL
    [CODE]

    Jun 26 10:00:56 fog rpc.mountd[23650]: authenticated mount request from IP.OF.YOUR.CLIENT:720 for /images (/images)
    [/CODE]
    The log file location (and message) might be slightly different (depending on your distribution), but running “ls -lhrt /var/log” should tell you which log file was most recently changed.

    For the disk image sizes, it looks like those are partimage (fog < 1.0) images. This is a known problem, see[URL=‘http://fogproject.org/forum/threads/disk-information-and-image-size-on-server-not-showing-any-data.10953/’] this thread[/URL] for more details on it.



  • [quote=“Jamie Rozek, post: 31440, member: 24394”]I’ve reset the fog user password on the server, thinking that could be the issue. I changed it to match the password in the error in my previous post.

    I am still getting the error:

    mount: mounting 192.168.3.8:/fog/images/ on /images failed: Permission denied.[/quote]

    I tried to FTP into the server from a command prompt on my Windows box. I was able to get in using that username/password.



  • I’ve reset the fog user password on the server, thinking that could be the issue. I changed it to match the password in the error in my previous post.

    I am still getting the error:

    mount: mounting 192.168.3.8:/fog/images/ on /images failed: Permission denied.



  • Okay. I just tried to delete one of the images off of the server.

    I received:

    [CENTER][FONT=Ubuntu][COLOR=#555555]FOGFTP: Login failed. Host: 192.168.3.8, [/COLOR][/FONT][/CENTER]
    [CENTER][FONT=Ubuntu][COLOR=#555555]Username: fog, Password: xxxxxx, Error:[/COLOR][/FONT][/CENTER]
    [CENTER][FONT=Ubuntu][COLOR=#555555] ftp_login(): Login incorrect.[/COLOR][/FONT][/CENTER]

    [CENTER][FONT=Ubuntu][COLOR=#555555]So, we have an FTP login problem somewhere. [/COLOR][/FONT][/CENTER]

    So, I check my GUI config:

    FOG_TFTP_FTP_PASSWORD is the same as the password listed in the above error message.

    I check Storage Management:

    [CENTER][FONT=Ubuntu][COLOR=#555555]Image path: /fog/images[/COLOR][/FONT][/CENTER]

    [CENTER][FONT=Ubuntu][COLOR=#555555]Management Password: Same as the password listed in the above error message.[/COLOR][/FONT][/CENTER]

    [CENTER][FONT=Ubuntu][COLOR=#555555]I check the Config.class.php file:[/COLOR][/FONT][/CENTER]

    TFTP_FTP_PASSWORD – matches.
    STORAGE_FTP_PASSWORD - matches.

    So, am I forgetting a config file somewhere or something? What is going on?
    [CENTER][FONT=Ubuntu][COLOR=#555555][/COLOR][/FONT][/CENTER]

    [CENTER][FONT=Ubuntu][COLOR=#555555][/COLOR][/FONT][/CENTER]



  • It is not allowing me to post my RPCinfo. States that it is “spam.”

    I’ve attached it as a text document.

    I’m seriously thinking about just doing a complete uninstall/reinstall.

    I’ve noticed that while this is happening, some of my images are doing this: (Every image I try to use, because “No size available” on the client.)

    [SIZE=5][B][FONT=Ubuntu][CENTER][SIZE=18px][COLOR=#4c68a0]All Images[/COLOR][/SIZE][/CENTER][/FONT][/B][/SIZE]

    [SIZE=13px][FONT=Ubuntu][COLOR=#555555][COLOR=#ededed]Image Name [CENTER]
    Storage Group[/CENTER]
    O/S [CENTER]
    Image Size: ON CLIENT[/CENTER] [CENTER]
    Image Size: ON SERVER[/CENTER] [CENTER]
    Uploaded[/CENTER] [CENTER] [/CENTER][/COLOR]
    [URL=‘http://192.168.3.8/fog/management/index.php?node=images&sub=edit&id=15’][COLOR=#7baa0f]2540P-Sales[/COLOR][/URL] [CENTER]default[/CENTER] Windows 7 [CENTER]223.47 GiB[/CENTER] [CENTER]80.54 GiB[/CENTER] [CENTER]10th, 3:19pm[/CENTER] [CENTER] [/CENTER]
    [URL=‘http://192.168.3.8/fog/management/index.php?node=images&sub=edit&id=17’][COLOR=#7baa0f]2570P-Win7ProBase[/COLOR][/URL] [CENTER]default[/CENTER] Windows 7 [CENTER]110.57 GiB[/CENTER] [CENTER]10.38 GiB[/CENTER] [CENTER]16th, 3:55pm[/CENTER] [CENTER] [/CENTER]
    [URL=‘http://192.168.3.8/fog/management/index.php?node=images&sub=edit&id=16’][COLOR=#7baa0f]6005-Base[/COLOR][/URL] [CENTER]default[/CENTER] Windows 7 [CENTER]No size available.[/CENTER] [CENTER]13.74 GiB[/CENTER] [CENTER]6th, 10:19pm[/CENTER] [CENTER] [/CENTER]
    [URL=‘http://192.168.3.8/fog/management/index.php?node=images&sub=edit&id=12’][COLOR=#7baa0f]e5540[/COLOR][/URL] [CENTER]default[/CENTER] Windows 8 [CENTER]223.57 GiB[/CENTER] [CENTER]102.12 GiB[/CENTER] [CENTER]20th, 12:25am[/CENTER] [CENTER] [/CENTER]
    [URL=‘http://192.168.3.8/fog/management/index.php?node=images&sub=edit&id=18’][COLOR=#7baa0f]E5540-MODPart[/COLOR][/URL] [CENTER]default[/CENTER] Windows 8 [CENTER]No size available.[/CENTER] [CENTER]10.60 GiB[/CENTER] [CENTER]17th, 6:30pm[/CENTER] [CENTER] [/CENTER]
    [URL=‘http://192.168.3.8/fog/management/index.php?node=images&sub=edit&id=19’][COLOR=#7baa0f]ModifiedImageE5540[/COLOR][/URL] [CENTER]default[/CENTER] Windows 8 [CENTER]No size available.[/CENTER] [CENTER]0.00 GiB[/CENTER] [CENTER]No Data[/CENTER] [CENTER] [/CENTER]
    [URL=‘http://192.168.3.8/fog/management/index.php?node=images&sub=edit&id=14’][COLOR=#7baa0f]Win7Pro[/COLOR][/URL] [CENTER]default[/CENTER] Windows 7 [CENTER]No size available.[/CENTER] [CENTER]5.34 GiB[/CENTER] [CENTER]6th, 6:18pm[/CENTER] [CENTER] [/CENTER][/COLOR][/FONT][/SIZE]

    [url="/_imported_xf_attachments/1/1075_rpcinfo.txt?:"]rpcinfo.txt[/url]


  • Testers

    You shouldn’t need to reinstall, it sounds like just an NFS problem. NFS(3) is pretty simple so we should be able to debug the problem here.

    Which distribution are you running? “service nfs restart” should work for RHEL based distributions but on ubuntu you might do something like “service nfs-kernel-server restart”. Alternatively you can just quickly reboot the fog server and let that restart the nfs service if you’re not sure. When it comes back up could you include the output from rpcinfo, e.g.
    [CODE]
    $ rpcbind -p
    program vers proto port service
    100000 4 tcp 111 portmapper
    100000 3 tcp 111 portmapper
    100000 2 tcp 111 portmapper
    100000 4 udp 111 portmapper
    100000 3 udp 111 portmapper
    100000 2 udp 111 portmapper
    100024 1 udp 48324 status
    100024 1 tcp 60634 status
    100011 1 udp 875 rquotad
    100011 2 udp 875 rquotad
    100011 1 tcp 875 rquotad
    100011 2 tcp 875 rquotad
    100005 1 udp 51151 mountd
    100005 1 tcp 56945 mountd
    100005 2 udp 38168 mountd
    100005 2 tcp 56201 mountd
    100005 3 udp 49959 mountd
    100005 3 tcp 41228 mountd
    100003 2 tcp 2049 nfs
    100003 3 tcp 2049 nfs
    100003 4 tcp 2049 nfs
    100227 2 tcp 2049 nfs_acl
    100227 3 tcp 2049 nfs_acl
    100003 2 udp 2049 nfs
    100003 3 udp 2049 nfs
    100003 4 udp 2049 nfs
    100227 2 udp 2049 nfs_acl
    100227 3 udp 2049 nfs_acl
    100021 1 udp 57018 nlockmgr
    100021 3 udp 57018 nlockmgr
    100021 4 udp 57018 nlockmgr
    100021 1 tcp 48912 nlockmgr
    100021 3 tcp 48912 nlockmgr

    100021    4   tcp  48912  nlockmgr
    

    [/CODE]

    Once you are sure that the services are running and listening on the correct ports, that really only leaves things that can terminate inbound connections like tcpwrappers.



  • I would really rather not uninstall the entire thing and start from scratch. I have some good “Golden images” on there.

    By the way, when I do:

    service nfs restart

    I get “unknown service.”



  • Yes, I’m pretty sure there is a setting that is different on the Fog server between versions 1.1.0 and 1.1.1/1.1.2, because I did not have this problem until upgrading to 1.1.1.
    I just re-installed Fog, minus the database—still having the same issue.

    I did not change a single thing. I don’t touch the server unless it is for Fog. We have no other linux servers.
    Selinux is disabled.


  • Testers

    It looks like the problem is between the clients and your fog server then, first could you check that you aren’t using selinux, e.g.
    [CODE]
    $ sestatus
    SELinux status: enabled
    SELinuxfs mount: /selinux
    Current mode: permissive
    Mode from config file: permissive
    Policy version: 24

    Policy from config file:        targeted
    

    [/CODE]

    Next, make sure that you aren’t using tcp_wrappers or configure it to allow rpcbind, portmap, statd etc. if you are.

    Do you have any other linux clients on your network that you could try the NFS mount from? If not you can schedule a debug task on one of the clients and run the mount commands there, it might give you a better idea of where the problem is. e.g.
    [CODE]
    $ mount -vvv -o vers=3,rw IP.OF.YOUR.FOG:/images /mnt
    [/CODE]



  • [quote=“ianabc, post: 31328, member: 24548”]It is worth checking your firewall and restarting the nfs service(s). Restarting the NFS service(s) depends on your system, but on mine looks like.
    [CODE]
    $ service nfs restart
    [/CODE]
    Also, could you send the output of exportfs -va, e.g.
    [CODE]
    $ exportfs -ua
    $ exportfs -va
    [/CODE]

    Also, another good test is to see if you can mount the NFS share on the server itself under a temporary location, e.g.
    [CODE]
    $ mount IP.OF.YOUR.FOG:/images /mnt
    $ ls /mnt
    dev Mint17x64PIMS1 U1404GAH87NWIFIIAM W7x32Stat55GBDeploy-XXX
    fog-math RHL6x64PIMS50GB u1404PIMS

    lost+found  RHL7x64BetaPIMS1  W7x32Stat55GBDeploy
    

    *** If it worked, remember to unmount it ****
    $ umount /mnt
    [/CODE][/quote]

    Firewall definitely not the issue. We haven’t changed anything, and Fog was working perfectly before the update yesterday.

    Here is the output of exportfs -va:

    exportfs: /etc/exports [1]: Neither ‘subtree_check’ or ‘no_subtree_check’ specified for export “*:/images”.
    Assuming default behaviour (‘no_subtree_check’).
    NOTE: this default has changed since nfs-utils version 1.0.x

    exportfs: /etc/exports [2]: Neither ‘subtree_check’ or ‘no_subtree_check’ specified for export “*:/images/dev”.
    Assuming default behaviour (‘no_subtree_check’).
    NOTE: this default has changed since nfs-utils version 1.0.x

    exporting *:/images/dev
    exporting *:/images

    I am able to mount IP.Address:/images.


  • Testers

    It is worth checking your firewall and restarting the nfs service(s). Restarting the NFS service(s) depends on your system, but on mine looks like.
    [CODE]
    $ service nfs restart
    [/CODE]
    Also, could you send the output of exportfs -va, e.g.
    [CODE]
    $ exportfs -ua
    $ exportfs -va
    [/CODE]

    Also, another good test is to see if you can mount the NFS share on the server itself under a temporary location, e.g.
    [CODE]
    $ mount IP.OF.YOUR.FOG:/images /mnt
    $ ls /mnt
    dev Mint17x64PIMS1 U1404GAH87NWIFIIAM W7x32Stat55GBDeploy-XXX
    fog-math RHL6x64PIMS50GB u1404PIMS

     lost+found  RHL7x64BetaPIMS1  W7x32Stat55GBDeploy
    

    *** If it worked, remember to unmount it ****
    $ umount /mnt
    [/CODE]


Log in to reply
 

438
Online

39.3k
Users

11.0k
Topics

104.6k
Posts

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.