Allow configuration for separate Imaging server and NFS server

rayhaque · Jun 16, 2014, 4:58 PM

Hello Tom, thank you for the feedback! Perhaps this will be easier to fix than I thought.

[quote=“Tom Elliott, post: 30434, member: 7271”]We are using a class based ftp function to do ftp now, but essentially the difference you need is about the root share of the folder. This is only affecting the NAS boxes you have.

Basically the NFS works properly as it’s understanding of the
/volume1/images share.
[/quote]

In my case, I have an /images directory on my FOG server which is actually an NFS mount.

[CODE]root@KCHIMAGE2:/# mount | grep nfs
10.205.205.20:/imaging_filesystem on /images type nfs (rw,rsize=8192,wsize=8192,timeo=14,intr,addr=10.205.205.20)
nfsd on /proc/fs/nfsd type nfsd (rw)
[/CODE]

For my DefaultMember storage node, the path is /imaging_filesystem. And the IP address is 10.205.205.20.

[quote=“Tom Elliott, post: 30434, member: 7271”]
FTP is failing on these boxes because it removes the /volume1/images part. When you connect it makes the “root” as /images.

So your FTP aren’t working properly. This isn’t something I can fix. While adding a “checkbox” for Synalogy/NAS setup would help you out, I imagine there’s actually a simpler configuration setting that can be performed on the NAS Boxes.[/quote]

Are you saying that the client is writing the file out to /imaging_filesystem/dev/filename okay - but then trying to rename it (using FTP) as if it exists in /imaging_filesystem/images/dev/filename? In other words, are you suggesting that this is a mount point issue?

rayhaque · Jun 17, 2014, 6:03 PM

I think I may be doing a lousy job of explaining the problem at hand. Or I was confused about what the problem was. Either way, here is a shorter explanation of what is happening.

An upload task is started, using the DefaultMember StorageNode, which is my SAN, 10.205.205.20 and a path of /imaging_filesystem.
Client connects directly to the SAN, using the magic of NFS and copies the data at high speeds.
The image upload works fine, and finishes.
Client attempts to make an FTP connection[B] to the SAN[/B] to rename the file. My SAN is not running FTP, nor does it need to!

Prior to the upgrade, I had made this possible by altering the code to make FOG look to itself instead of the SAN when it got to the FTP elements. On my FOG server, I have an NFS mount to 10.205.205.20 named /imaging_filesystem - which I have symbolically linked to /images. So if it can manage to FTP back to itself, we are in good shape. Because it can reach out to /imaging_filesystem/dev/* or /images/dev/* (doesn’t matter, they are one in the same).

I think I may have found the code that I need to alter, and I am testing it now. Likely, my upload will not complete until I have left for the day, but I will report my progress later for anyone who cares.

My code change is to /var/www/fog/service/Post_Stage2.php, lines 53 and 54:
[CODE] //Attempt transfer of image file to Storage Node
$ftp->set(‘host’,$StorageNode->get(‘ip’))[/CODE]
Becomes …
[CODE] //Attempt transfer of image file to Storage Node
$ftp->set(‘host’,$this->FOGCore->getIPAddress())[/CODE]

Tom Elliott · Jun 17, 2014, 6:12 PM

Your situation is quite different, but this should fix it for you.

Hopefully I helped, but this isn’t the “normal” operational mode of things expected and is fairly on a “higher” technical scale than I could have appropriately guessed somebody would try to do. I appreciate the feedback as well and hope the best with this. It should work perfectly fine now that I understand exactly what you’re trying to do.

I have a similar setup, but rather than a pure NFS connection to the storage server, I’m using Glusterfs to mount as it’s own point, and using the actual FOG Server to be the Node AND the NFS share.

rayhaque · Jun 17, 2014, 6:59 PM

[quote=“Tom Elliott, post: 30566, member: 7271”]Your situation is quite different, but this should fix it for you.[/quote]

I think I am close! But I get this error …
[CODE][Tue Jun 17 14:56:49 2014] [error] [client 10.10.5.16] PHP Fatal error: Using $this when not in object context in /var/www/fog/service/Post_Stage2.php on line 54[/CODE]

I am not sure how I can refer to FOGCore in that context? I am not a very good programmer. Any ideas?

rayhaque · Jun 17, 2014, 7:02 PM

Incidentally, if I just bang the IP address in there, it works fine!

Like so …
[CODE]$ftp->set(‘host’,‘10.10.5.10’)[/CODE]

Tom Elliott · Jun 17, 2014, 7:04 PM

This InContext usually refers to a class file where the post_stage III and stage two files or any of the service files for that matter are not class files if you take your get IP address function and just called dollar fog core you’ll be good to go

Tom Elliott · Jun 17, 2014, 7:43 PM

Sorry about my last post, I was on google translate so it probably made no sense.

The problem is this right here:
[CODE] //Attempt transfer of image file to Storage Node
$ftp->set(‘host’,$this->FOGCore->getIPAddress())[/CODE]
It should read as:
[code] //Attempt transfer of image file to Storage Node
$ftp->set(‘host’,$FOGCore->getIPAddress())[/code]

rayhaque · Jun 17, 2014, 9:30 PM

Thanks Tom,

I know that I didn’t try that syntax. I will try to make some time tomorrow and test this code snippet.

ianabc · Jun 20, 2014, 8:27 PM

If this works, is there a systematic way to have this be an option in the UI? It would be a really nice feature for me and a couple of other people. In general we have NFS storage nearby that I would like to use but which I can’t turn into a full fledged fog storage node.

rayhaque · Feb 20, 2015, 2:27 PM

I am upgrading from 1.1.1 to 1.2.0 and I can see that I will have this same problem again. Again, you would need to do this in the case that your NFS server (the FOG server) is not the same as where your files are (on a NAS or SAN). I wrote up this patch file. You would want to change 10.205.205.20 to the IP address of your SAN/NAS and you would want to change [FONT=Consolas]10.10.5.10 to the IP address of your FOG server.[/FONT]

[CODE]— packages/web/service/Post_Stage2.php.orig 2015-02-20 08:59:27.731899437 -0500
+++ packages/web/service/Post_Stage2.php 2015-02-20 08:59:51.191898383 -0500
@@ -52,7 +52,7 @@
// Where is it going?
$dest = $StorageNode->get(‘path’).‘/’.$_REQUEST[‘to’];
//Attempt transfer of image file to Storage Node

                 $ftp->set('host',$StorageNode->get('ip'))

                 $ftp->set('host','10.10.5.10')
                          ->set('username',$StorageNode->get('user'))
                          ->set('password',$StorageNode->get('pass'));
                  if (!$ftp->connect())

— packages/web/lib/fog/ImageReplicator.class.php.orig 2015-02-20 09:02:09.363892178 -0500
+++ packages/web/lib/fog/ImageReplicator.class.php 2015-02-20 09:03:14.135889269 -0500
@@ -16,7 +16,7 @@
}
private function commonOutput()
{

         $StorageNode = current($this->FOGCore->getClass('StorageNodeManager')->find(array('isMaster' => 1,'isEnabled' => 1, 'ip' => $this->FOGCore->getIPAddress())));

         $StorageNode = current($this->FOGCore->getClass('StorageNodeManager')->find(array('isMaster' => 1,'isEnabled' => 1, 'ip' => '10.205.205.20')));
          try
          {
          if ($StorageNode)

— packages/web/lib/fog/MulticastManager.class.php.orig 2015-02-20 09:04:44.975885190 -0500
+++ packages/web/lib/fog/MulticastManager.class.php 2015-02-20 09:05:23.587883456 -0500
@@ -78,7 +78,7 @@
{
try
{

                         $StorageNode = current($this->FOGCore->getClass('StorageNodeManager')->find(array('isMaster' => 1,'isEnabled' => 1,'ip' => $this->FOGCore->getIPAddress())));

                         $StorageNode = current($this->FOGCore->getClass('StorageNodeManager')->find(array('isMaster' => 1,'isEnabled' => 1, 'ip' => '10.205.205.20')));
                          if (!$StorageNode || !$StorageNode->isValid())
                                  throw new Exception(sprintf(" | StorageNode Not found on this system."));
                          $myroot = $StorageNode->get('path');

— packages/web/lib/fog/Image.class.php.orig 2015-02-20 09:06:19.163880960 -0500
+++ packages/web/lib/fog/Image.class.php 2015-02-20 09:07:02.043879034 -0500
@@ -159,7 +159,7 @@
$SNME = ($SN && $SN->get(‘isEnabled’) == ‘1’ ? true : false);
if ($SNME)
{

                 $ftphost = $SN->get('ip');

                 $ftphost = '10.10.5.10';
                  $ftpuser = $SN->get('user');
                  $ftppass = $SN->get('pass');
                  $ftproot = rtrim($SN->get('path'),'/').'/'.$this->get('path');

— packages/web/lib/fog/FOGFTP.class.php.orig 2015-02-20 09:07:59.171876468 -0500
+++ packages/web/lib/fog/FOGFTP.class.php 2015-02-20 09:08:28.223875164 -0500
@@ -28,7 +28,7 @@
if (($this->link && $this->lastConnectionHash == $connectionHash) || !$this->get(‘host’) || !$this->get(‘username’) || !$this->get(‘password’) || !$this->get(‘port’))
return $this;
// Connect

         $this->link = @ftp_connect($this->get('host'), $this->get('port'), $this->get('timeout'));

         $this->link = @ftp_connect(('10.10.5.10'), $this->get('port'), $this->get('timeout'));
          if (!$this->link)
          {
                  $error = error_get_last();

[/CODE]

Apply it like this …
Save the above file to fog_1.2.0.patch, and then:
[CODE]
tar xzf fog_1.2.0.tar.gz
cp ~/fog_1.2.0.patch fog_1.2.0
cd fog_1.2.0
patch -p0 < fog_1.2.0.patch
[/CODE]

Allow configuration for separate Imaging server and NFS server

173

12.0k

17.3k

155.2k