Solved Image hosting on NFS server, process uncompleted?

[arnaudrigole]

Hi Fog Community,

Again an NFS issue…
I try to upload an image on a external NFS share, the process starts, upload the image on mounted /images/dev directory, the two “diskclones” process work and create /dev/mac@ddr_of_master/d1.mbr , d1p1.img, d1p2.img, d1p3.img.
At the end, i think it has to create in /images a directory named by image name defined on webui, and transfer that files into it. But it won’t … the .img keep in /dev and the upload process finish without errors

Any ideas?

[Wayne Workman]

@Wayne-Workman said in Image hosting on NFS server, process uncompleted?:

@arnaudrigole I have some methods documented.

using lftp:
lftp -c 'open x.x.x.x; user UserGoesHere PasswordHere; mirror -e /images/TheImagePath /images/TheImagePath; quit'

using NFS:
mount x.x.x.x:/<remote directory> <local directory>
cp -R /mbimages/Optiplex9020UEFI /images/Optiplex9020UEFI
umount <local directory>

Using Samba:
mkdir /tempMount
mount -t cifs //x.x.x.x/ShareNameGoesHere /tempMount -o username=YourUsernameGoesHere -o password=YourPasswordGoesHere,noexec
cp -r /tempMount/ImageNameHere /images/ImageNameHere
umount /tempMount

using SCP:
scp -r /images/ImageNameHere root@x.x.x.x:/images/ImageNameHere

using rsync:
rsync -a /images/ImageNameHere root@x.x.x.x:/images/ImageNameHere

wiki worthy

Added to the wiki here:
https://wiki.fogproject.org/wiki/index.php?title=Migrate_images_manually

[Wayne Workman]

@arnaudrigole said in Image hosting on NFS server, process uncompleted?:

What is FOG Replication?

The below article is about the location plugin, but contains information about replication.
https://wiki.fogproject.org/wiki/index.php?title=Location_Plugin

[arnaudrigole]

@Wayne-Workman
Hi Wayne,
What is FOG Replication?

I followed the procedure to test FTP, it was a success. I could connect with fog credentials, create and transfer the test.txt file on external /images directory, and download then delete it…

@george1421
Filezilla displays the logons and actions directly in its console, the “logs” are on page 2, “directory not found” is displayed, because the correspondant directory is not created by fog…

What is the part of fog upload script which create that directories in /images ? i’m almost certain thats component is not working, maybe it has no rw rights on /images , but i don’t know which user / process / protocol is used to create it (FTP too ?)

[Wayne Workman]

@arnaudrigole said in Image hosting on NFS server, process uncompleted?:

I just transfered .img files in a directory named “P7510Win7prox64CAO” , exactly like defined in the “image path” on the image i created on fog webgui.

You shouldn’t have had to do the transfer manually. FOG Replication will take care of this - if - FTP is setup correctly.

As a test, from your FOG server, you can try to FTP into the windows storage node using commands found here:
https://wiki.fogproject.org/wiki/index.php?title=Troubleshoot_FTP#Try_to_get_a_file_with_Linux:

[george1421]

ok lets assume that all FTP is equal.

Does filezilla have a log file so you can check to see if the FOG server is actually connecting to it? If there is a way to log what the FOG server is doing it would help us debug this.

Understand this is outside of the standard setup so a lot of this is just guessing.

[arnaudrigole]

@george1421
Thanks for that precisions. The credentials used are both the same on webgui & filezilla.

I don’t use built-in ftp server because the procedure described is for 2012, and i don’t master 2008 powershell anymore to find equivalent cmds … FileZilla looks more simply, and seems working because i can connect on it from any client, write any file or directory on it with no problem. But FOG can’t…

[george1421]

@arnaudrigole The fog server talks to the storage nodes (windows or linux).

IF the storage node is a linux server with the fog storage node install this communication is via http calls.

If the storage node doesn’t respond with the proper response to the http calls it will revert back to the old way and use FTP to communicate with the storage node. You need to ensure the ftp user ID and password defined in the storage node configuration matches the ftp user account on the remote storage node. Getting the image size IS a function of the FTP communications. The home server for the FOG FTP account must be the images root on the remote storage node. If this is wrong then the FTP communications won’t work.

[arnaudrigole]

@george1421 @Wayne-Workman
Wayne, remember , i’ve already set the complete environment NFS / FTP on my windows server using filezilla & windows nfs . Fog user is already created, with full rw rights on root E: E:/images E:/images/dev

@george1421 I don’t understand what you say ? my ftp server is set up and work ?

[george1421]

@arnaudrigole Sorry I’ve been involved with too many threads in the last few days.

For the image size bit, you need to have an ftp server setup on that windows storage node since if the storage node doesn’t respond to the http calls as with a normal linux storage node, Tom set it up to default back to the FTP query to get the image size.

[edit]
Well if Wayne can post a link, so can I: https://forums.fogproject.org/topic/6941/windows-server-as-fog-storage-node-reboot The first post there gives the powershell commands to use the windows built in ftp server for FOG.

[Wayne Workman]

@arnaudrigole You need to setup FTP on the windows node, create an FTP account called fog, and then give that account full access to the images directory.

Also: https://forums.fogproject.org/topic/6941/windows-server-as-fog-storage-node-reboot

[arnaudrigole]

@Wayne-Workman
I just transfered .img files in a directory named “P7510Win7prox64CAO” , exactly like defined in the “image path” on the image i created on fog webgui.

Created a storage group for my windows server, and setup this image on that storage group.

Fog don’t see the image size, i think it dont recognize it…

[Wayne Workman]

@arnaudrigole Oh. I guess I missed the part about the windows storage node somewhere? Sorry.

Guess Sebastian was right to begin with (he doesn’t make mistakes too often).

So, if the windows server is configured as a storage node, you’re good to go. Don’t worry about the image definitions.

However - You’ll need more than NFS on the windows server, you’ll need FTP too. Windows Server comes with both of these things native, and I would recommend using the windows native tools to set it up.

@george1421 has a writeup explaining how to do it all via PowerShell - and I would recommend his instructions. In the past, I’ve tried setting it up via GUI in windows and things just weren’t working.

However… I do now have a windows server setup at home, it could be an opportune time to maybe document the process…

[arnaudrigole]

@george1421 @Sebastian-Roth @Wayne-Workman

Thanks for your help. I haven’t new fog server, i just setup a new storage node on Windows Server 2008R2.

Currently, my images are stored on /images on the fog local server, i wanna know if it’s possible to move em on my windows nfs server, like Wayne described : copy all the .img files in fogserver:/images in a directory named exactly as the local fog server directory.

I’m currently copying an image (18gb) on nfs server, then i’ll create the correspondant entry on fog webui and will see if it see the image : )

[george1421]

Or depending on the source FOG server version, export the Image definitions into a csv file and then import them into the new fog server. The image files are already on the target server, what is missing is the db definitions.

We have to do something similar in our multi-master/storage node setup (unique configuration for our application. But only when we add new images in our environment).

[Sebastian Roth]

Ohhhh, didn’t read properly. If this is a complete new server (and DB) then you need to create the image definition or import those from a DB dump… sorry for that!

[Wayne Workman]

@Sebastian-Roth I think he’s talking about the image definitions, in the web gui.

If the image definitions are not already there, and the old server was the same version as the new one, you can try to export the images and import them. Otherwise you must create the definitions manually.

[arnaudrigole]

@Sebastian-Roth
Ok i’ll try :). Thank you.

[Sebastian Roth]

@arnaudrigole When you put the files in the correct directory FOG will see them. No need to index or anything!

[arnaudrigole]

@Wayne-Workman
Hello Wayne,
After i’ve transfered all img files on the new location (nfs server), how to index it on fog webgui? (make the image recognized by fog server)