Permission Denied when trying to image after update to 1.1.1/1.1.2



  • Yesterday I updated my Fog server to 1.1.1. When trying to image, I now get the following: Capture.PNG



  • Thanks, guys. I finally got it working. It was a mess…I must have just started changing things willy-nilly and stopped paying attention to what I was doing.
    The export file, as ianabc stated was incorrect. I did:
    nano /etc/exports
    pointed the first line to fog/images *(ro,sync,no_wdelay,insecure_locks,no_root_sq$
    and my machines immediately started imaging.

    The funny thing is that yesterday I’m almost certain I had a line in that file pointing to that path. Maybe I didn’t comment out the other one.

    Either way, it appears to be working. Hopefully I didn’t change something else somewhere that’s going to drive me nuts!

    Thanks!!!


  • Testers

    Hi Jamie, I just noticed in your syslog

    
    refused mount request from 192.168.3.33 for /fog/images (/): not exported
    
    

    The fog clients are trying to mount the wrong location, that is almost certainly the problem. Check the image path in your storage configuration, you would either need to update that or change your export list, once they match your NFS problem should go away (and probably the FTP one as well).


  • Senior Developer

    What are the permissions of /images?

    Can you try these steps:

    touch /images/{,dev/}.mntcheck
    chmod -R 777 /images
    service nfs-kernel-server restart
    

  • Testers

    
    Jun 26 13:22:56 troi rpc.mountd[1691]: authenticated mount request from 192.168.3.33:930 for /images (/images)
    
    

    Does this correspond with the fog debug task and mount command? did you get any output from that command? Could you turn on debugging for NFS, edit the /etc/default/nfs-kernel-server file to have the line

    
    RPCMOUNTDOPTS="--manage-gids --debug all"
    
    

    Then “service nfs-kernel-server restart” and retry the mount - syslog should get more information this time.


  • Testers

    OK, so it looks like syslog would be the where NFS mounts are logged on your system, is it Ubuntu? I have an ubuntu machine image and I’m deploying it now. Once it is installed I’ll take a look at the NFS setup so we can compare. I’ll post back when it is up and running.

    When I said I thought the network was the problem I meant the networking on the fog server you are running: the installer script runs yum updates and my guess is that a new version of one of these packages broke something.



  • Output of the SYSLOG: (It is refusing the mount request)

    (Attached----it was reporting back as Spam when trying to post)

    syslog.txt



  • Output of ls -lhrt /var/log:

    drwxr-xr-x 2 root root 4.0K Jun 26 07:03 unattended-upgrades
    -rw-r–r-- 1 root root 235K Jun 26 07:08 dpkg.log
    -rw-r----- 1 mysql adm 0 Jun 26 07:08 mysql.log
    -rw-r----- 1 root adm 0 Jun 26 07:08 apport.log
    drwxr-s— 2 mysql adm 4.0K Jun 26 07:08 mysql
    -rw-r----- 1 syslog adm 1.3M Jun 26 07:08 syslog.1
    drwxr-xr-x 2 root root 4.0K Jun 26 07:08 upstart
    -rw-r----- 1 syslog adm 410K Jun 26 09:30 kern.log
    -rw-r----- 1 syslog adm 5.3K Jun 26 13:22 syslog
    -rw-r----- 1 syslog adm 72K Jun 26 13:23 auth.log
    -rw-rw-r-- 1 root utmp 113K Jun 26 13:23 wtmp
    -rw-rw-r-- 1 root utmp 429K Jun 26 13:23 lastlog



  • this thread for more details on it.[/QUOTE]

    Nope. They have all been made on 1.1.0 over the last few weeks.

    As for the rest being a connection problem—I don’t see how. We haven’t changed a thing on our network. The ONLY thing that has changed is that we updated the Fog server. That’s it. It was upgraded to 1.1.1, and when that didn’t work – 1.1.2.
    It was working just fine on 1.1.0.


  • Testers

    The message this thread for more details on it.



  • @Jamie Rozek, post: 31440, member: 24394 said:

    I’ve reset the fog user password on the server, thinking that could be the issue. I changed it to match the password in the error in my previous post.

    I am still getting the error:

    mount: mounting 192.168.3.8:/fog/images/ on /images failed: Permission denied.

    I tried to FTP into the server from a command prompt on my Windows box. I was able to get in using that username/password.



  • I’ve reset the fog user password on the server, thinking that could be the issue. I changed it to match the password in the error in my previous post.

    I am still getting the error:

    mount: mounting 192.168.3.8:/fog/images/ on /images failed: Permission denied.



  • Okay. I just tried to delete one of the images off of the server.

    I received:

    [CENTER]%(#555555]FOGFTP)[[/CENTER]
    [CENTER]Username: fog, Password: xxxxxx, Error:][/CENTER]
    [CENTER]%(#555555] ftp_login())[[/CENTER]

    [CENTER]So, we have an FTP login problem somewhere. ][/CENTER]

    So, I check my GUI config:

    FOG_TFTP_FTP_PASSWORD is the same as the password listed in the above error message.

    I check Storage Management:

    [CENTER]%(#555555]Image path)[[/CENTER]

    [CENTER]Management Password: Same as the password listed in the above error message.][/CENTER]

    [CENTER]%(#555555]I check the Config.class.php file)[[/CENTER]

    TFTP_FTP_PASSWORD – matches.
    STORAGE_FTP_PASSWORD - matches.

    So, am I forgetting a config file somewhere or something? What is going on?
    [CENTER]][/CENTER]

    [CENTER][/CENTER]



  • It is not allowing me to post my RPCinfo. States that it is “spam.”

    I’ve attached it as a text document.

    I’m seriously thinking about just doing a complete uninstall/reinstall.

    I’ve noticed that while this is happening, some of my images are doing this: (Every image I try to use, because “No size available” on the client.)

       rpcinfo.txt

  • Testers

    You shouldn’t need to reinstall, it sounds like just an NFS problem. NFS(3) is pretty simple so we should be able to debug the problem here.

    Which distribution are you running? “service nfs restart” should work for RHEL based distributions but on ubuntu you might do something like “service nfs-kernel-server restart”. Alternatively you can just quickly reboot the fog server and let that restart the nfs service if you’re not sure. When it comes back up could you include the output from rpcinfo, e.g.

    
      $ rpcbind -p
       program vers proto   port  service
        100000    4   tcp    111  portmapper
        100000    3   tcp    111  portmapper
        100000    2   tcp    111  portmapper
        100000    4   udp    111  portmapper
        100000    3   udp    111  portmapper
        100000    2   udp    111  portmapper
        100024    1   udp  48324  status
        100024    1   tcp  60634  status
        100011    1   udp    875  rquotad
        100011    2   udp    875  rquotad
        100011    1   tcp    875  rquotad
        100011    2   tcp    875  rquotad
        100005    1   udp  51151  mountd
        100005    1   tcp  56945  mountd
        100005    2   udp  38168  mountd
        100005    2   tcp  56201  mountd
        100005    3   udp  49959  mountd
        100005    3   tcp  41228  mountd
        100003    2   tcp   2049  nfs
        100003    3   tcp   2049  nfs
        100003    4   tcp   2049  nfs
        100227    2   tcp   2049  nfs_acl
        100227    3   tcp   2049  nfs_acl
        100003    2   udp   2049  nfs
        100003    3   udp   2049  nfs
        100003    4   udp   2049  nfs
        100227    2   udp   2049  nfs_acl
        100227    3   udp   2049  nfs_acl
        100021    1   udp  57018  nlockmgr
        100021    3   udp  57018  nlockmgr
        100021    4   udp  57018  nlockmgr
        100021    1   tcp  48912  nlockmgr
        100021    3   tcp  48912  nlockmgr
     
        100021    4   tcp  48912  nlockmgr
    
    

    Once you are sure that the services are running and listening on the correct ports, that really only leaves things that can terminate inbound connections like tcpwrappers.



  • I would really rather not uninstall the entire thing and start from scratch. I have some good “Golden images” on there.

    By the way, when I do:

    service nfs restart

    I get “unknown service.”



  • Yes, I’m pretty sure there is a setting that is different on the Fog server between versions 1.1.0 and 1.1.1/1.1.2, because I did not have this problem until upgrading to 1.1.1.
    I just re-installed Fog, minus the database—still having the same issue.

    I did not change a single thing. I don’t touch the server unless it is for Fog. We have no other linux servers.
    Selinux is disabled.


  • Testers

    It looks like the problem is between the clients and your fog server then, first could you check that you aren’t using selinux, e.g.

    
      $ sestatus
        SELinux status:                enabled
        SELinuxfs mount:                /selinux
        Current mode:                  permissive
        Mode from config file:          permissive
        Policy version:                24
     
        Policy from config file:        targeted
    
    

    Next, make sure that you aren’t using tcp_wrappers or configure it to allow rpcbind, portmap, statd etc. if you are.

    Do you have any other linux clients on your network that you could try the NFS mount from? If not you can schedule a debug task on one of the clients and run the mount commands there, it might give you a better idea of where the problem is. e.g.

    
      $ mount -vvv -o vers=3,rw IP.OF.YOUR.FOG:/images /mnt
    
    


  • @ianabc, post: 31328, member: 24548 said:

    It is worth checking your firewall and restarting the nfs service(s). Restarting the NFS service(s) depends on your system, but on mine looks like.

    >
      $ service nfs restart
    
    >```
    Also, could you send the output of exportfs -va, e.g.
    

    $ exportfs -ua
    $ exportfs -va

    Also, another good test is to see if you can mount the NFS share on the server itself under a temporary location, e.g.

    >
      $ mount IP.OF.YOUR.FOG:/images /mnt
      $ ls /mnt
        dev        Mint17x64PIMS1    U1404GAH87NWIFIIAM  W7x32Stat55GBDeploy-XXX
        fog-math    RHL6x64PIMS50GB  u1404PIMS
     
        lost+found  RHL7x64BetaPIMS1  W7x32Stat55GBDeploy
     
      *** If it worked, remember to unmount it ****
      $ umount /mnt
    
    >```
     
     
     
    Firewall definitely not the issue.  We haven't changed anything, and Fog was working perfectly before the update yesterday.
     
    Here is the output of exportfs -va: 
     
    exportfs: /etc/exports [1]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/images".
      Assuming default behaviour ('no_subtree_check').
      NOTE: this default has changed since nfs-utils version 1.0.x
     
    exportfs: /etc/exports [2]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/images/dev".
      Assuming default behaviour ('no_subtree_check').
      NOTE: this default has changed since nfs-utils version 1.0.x
     
    exporting *:/images/dev
    exporting *:/images
     
     
    I am able to mount IP.Address:/images.

  • Testers

    It is worth checking your firewall and restarting the nfs service(s). Restarting the NFS service(s) depends on your system, but on mine looks like.

    
      $ service nfs restart
    
    

    Also, could you send the output of exportfs -va, e.g.

    
      $ exportfs -ua
      $ exportfs -va
    
    

    Also, another good test is to see if you can mount the NFS share on the server itself under a temporary location, e.g.

    
      $ mount IP.OF.YOUR.FOG:/images /mnt
      $ ls /mnt
         dev         Mint17x64PIMS1    U1404GAH87NWIFIIAM   W7x32Stat55GBDeploy-XXX
         fog-math    RHL6x64PIMS50GB   u1404PIMS
     
         lost+found  RHL7x64BetaPIMS1  W7x32Stat55GBDeploy
     
      *** If it worked, remember to unmount it ****
      $ umount /mnt
    
    

Log in to reply
 

370
Online

38719
Users

10548
Topics

99848
Posts

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.