Fog/Linux cant image, nfs and rpcbind issues

[neodawg]

@george1421

Sorry its not the the NFS process its the rpcbind process, NFS stays active until I restart it, which will fail if I dont restart rpcbind first.

[neodawg]

So for fun or insanity, i downgraded rpcbind.x86_64 0:0.2.0-13.el6_9 to rpcbind.x86_64 0:0.2.0-13.el6 on the CentOS 6 box I am local to and I think that may have fixed the issue. The image is currently uploading in debug mode, going to try pushing it out as soon as its done.

The other thing I haven’t ruled out is having open-vm-tools installed. I dont know why this would break things but I read something somewhere on some forum about it.

I did yum downgrade rpcbind on the CentOS 7 box and the services are still running, but the local tech hasnt tried imaging again yet.

Resolving Dependencies
--> Running transaction check
---> Package rpcbind.x86_64 0:0.2.0-38.el7 will be a downgrade
---> Package rpcbind.x86_64 0:0.2.0-38.el7_3 will be erased
--> Finished Dependency Resolution

EDIT: it appears this process fixed the CentOS 6 box, it was able to deploy the same image after an immediate capture of it.

[george1421]

@neodawg Sorry I was working on another issue. That is very strange.

If I get a chance to night I’ll spin up a new FOG server with a full upgrade. Its possible something was pushed out over the weekend causing nfs to fail.

[george1421]

@neodawg Do you have ipv6 disabled on this box?
ref: http://www.linuxquestions.org/questions/linux-software-2/cenos-7-3-rpcbind-4175596401/

[george1421]

I can’t duplicate your issue. Understand I’m not saying that you don’t have an issue. I just can’t duplicate it.

I setup a new 1.4.2 fog server on a fresh install of Centos 7. NFS started as it should.

Details of the build
VM build on ESXi 6.5
Centos 7 x64 v1611

I installed centos minimal
yum upgrade -y
set selinux permissive
systemctl disable firewalld
reboot
git clone https://github.com/FOGProject/fogproject.git /opt/fogproject
cd /opt/fogproject/bin
./installfog.sh
(installed completed without issue)

Installed version of rpcbind: rpcbind-0.2.0-38.el7_3.x86_64
Installed version of open-vm-tools: open-vm-tools-10.0.5-4.el7_3.x86_64

Even after a reboot nfs and rpcbind services are still happy.

[Sebastian Roth]

This sounds similar… https://bugzilla.redhat.com/show_bug.cgi?id=1457963

@george1421 Maybe you need to have a client mount the share to run into the same issue?!?

[neodawg]

@Sebastian-Roth and @george1421

Yes, the service will start and run, but shortly after a client tries to connect the service will die.

The link you listed sound exactly what is happening, however i didn’t test different versions of NFS protocol, ie v3 vs v4.

On one of the fog servers the tech was able to actually image one computer successfully and then the next computer it failed on, because the rpcbind service had died.

[neodawg]

I heard back from the local tech with the CentOS 7 box and downgrading rpcbind fixed the issue on that server as well. I guess we will have to wait until RedHat/CentOS fixes the rpcbind package

[Sebastian Roth]

@george1421 I just found some more reports, e.g. https://bugzilla.redhat.com/show_bug.cgi?id=1448124
Seems like they are working on it. Let’s hope they are able to fix this fairly soon! See here: https://bugzilla.redhat.com/show_bug.cgi?id=1457172

[george1421]

@Sebastian-Roth Great find!! I still have that test system setup. I’ll update dhcp and attempt to image a VM a system this morning. There was no time earlier this weeek to confirm once a client tries to image nfs fails. I will have a few minutes today to test. Just thinking if its a memory leak it issue may not show up right away. I guess we will find out.

[Sebastian Roth]

Checking the CentOS repo just now I saw a new package being available since 13th of June - search for rpcbind here. Anyone keen to test? @neodawg