Solved Error returned: Type: 1024, File: /var/www/fog/lib/fog/fogftp.class.php
It’s the second week and I can’t seem to find where the error is. I have activated debug mode on FOS but still:
* Stopping FOG Status Reporter......................Done * Press [Enter] key to continue * Task Complete * Updating Database.................................Failed * Press [Enter] key to continue * Error returned: Type: 1024, File: /var/www/fog/lib/fog/fogftp.class.php, Line : 219, Message: FTP connection failed, Host: xx.xxxxxx.xxx, Username: fogprojec t * Press [Enter] key to continue
I have checked everything as in the wiki
I am able to connect with telnet on port 21 from FOS and also from another host with same credentials. I can move, add, rename, etc. files and folders from /images with ftp user fogproject.
We have multiple storage nodes, this is the only one with this problem. All others work fine.
I checked the logs on the master and on this storage node and I can’t seem to find anything.
Any ideeas or hints where to look for?
@gn_ro Lets have you run through this tutorial to see if we can get things back in sync. https://forums.fogproject.org/topic/11203/resyncing-fog-s-service-account-password
This sometimes happens if the fog admin changes the password on the fogproject user or you have a half install of the software.
@george1421 Thanks, I did that before posting… same error.
What does fog means by updating database?
I can move files using a telnet session from FOS to the storage node:
[Thu Mar 31 root@fogclient ~]# telnet storagenode 21 Connected to storagenode 220 (vsFTPd 3.0.3) user fogproject 331 Please specify the password. pass xxxxxxxxxxxxx 230 Login successful. rnfr /images/dev/testftp 350 Ready for RNTO. rnto /images/ftp-tested 250 Rename successful.
But fog does not move the image after cloning succesfully.
@gn_ro Well the updating database bit is misleading.
At this point in the imaging process the FOS engine all of the raw image files have been sent to the FOG server. They are currently in the /images/dev directory on the master FOG server. Then FOS engine (running on the target computer) connects to the FOG server using FTP and issues a move command from /images/dev/<mac_address> to /images/<image_name> directory. It does this using FTP because in linux it doesn’t have to move the GBs of raw image files it only updates the file access table of the directory move. Its very fast this way. After the move has completed then it updates the database the move is complete. That is where its failing, not necessarily updating the database but moving the raw data files.
My bet is if you look at the /images/dev directory you will see a bunch of directories that look like mac addresses. Unless you are imaging right at this moment, those are botched uploads. And can be deleted to free up disk space.
So the question is why doesn’t the fogproject user have rights to move files between /images/dev and /images directory.
What are the outputs of these commands.
ls -la /images/dev ls -la /images
Second test see if you can ftp into the fog server using the user ID
fogprojectand the password found in
/opt/fog/.fogsettingsfile. Note the dot at the beginning of the name makes the file hidden in linux.
@gn_ro Well heck I should have read your second post where you did everything I just asked.
Is there any useful error messages in either /var/log/apache2/error.log (or any error log in there) or /var/log/php-fpm/error.log? There has to be a clue somewhere to why its throwing the error if the file can be moved by the fogproject user.
After FOS finished cloning with debug:
telnet storagenode 21 Connected to storagenode 220 (vsFTPd 3.0.3) user fogproject 331 Please specify the password. pass xxxxxxxxxxxxxx 230 Login successful. rnfr /images/dev/18dbf21b59d1 350 Ready for RNTO. rnto /images/to-del 250 Rename successful.
On the storage node:
root@st:/# ls -la /images | grep del drwxrwxrwx 2 fogproject root 11 Mar 26 11:25 to-del root@st:/# ls -la /images/dev total 10 drwxrwxrwx 7 fogproject root 8 Apr 2 00:05 . drwxrwxrwx 33 fogproject root 35 Apr 2 00:05 .. drwxrwxrwx 2 fogproject root 6 Apr 1 22:58 0019999f5919 drwxrwxrwx 2 fogproject root 11 Mar 26 11:35 18dbf24c71c8 drwxrwxrwx 2 fogproject root 6 Apr 1 23:47 704d7bbf8ebe drwxrwxrwx 2 fogproject root 6 Apr 1 23:52 704d7bbf8ef9 -rwxrwxrwx 1 fogproject root 0 Oct 30 12:14 .mntcheck drwxrwxrwx 2 fogproject root 3 Oct 30 12:14 postinitscripts root@st:/#
It’s clear that I can rename the cloned files to the ‘to-del’ folder, so FOS should be able to do this, but it’s not (the rename is not made after the cloning is done).
I’ve rechecked the passwords on the fog master and on this storage node. I’ve also used a backup of this node from 3 weeks ago (before the issue appeared) The FOS does not appear to be initiating the ftp connection at all, I’ve checked with tcpdump and no packet comes to the storage node from FOS.
I’m out of ideeas for now…
The FOS does not appear to be initiating the ftp connection at all, I’ve checked with tcpdump and no packet comes to the storage node from FOS.
Maybe a firewall between master and storage node? Be aware that the FTP connection is not made from the cloned host machine itself but from the master server.
@sebastian-roth @george1421 Solved! Thank you for your time!
There is no firewall between the host and storage node. There is a firewall between the storage node and the master but is correctly configured.
The problem was that the storage node is registered at the master with his dns name and this was somehow deleted from the domain.
I have restored the dns record for the storage node and now all is functions correctly except “Storage Node Disk Usage” from the dashboard (it shows nothing), but this is not a big problem.