Solved FOG won't move images

[Quazz]

Hi, I’ve been over the FTP troubleshooting guide, followed each step and confirmed that FTP is indeed working, correct passwords everywhere (I can copy paste the password from any location into the ftp client and it works), but images are unable to be moved.

The FTP config file is identical to the one on the wiki as well. Ownership and permissions should all be fine. (0777, fog:root)

As for the log files, I can’t see any of the FOG Service log files anymore.

[Sebastian Roth]

Good catch man! Marking this solved.

[Quazz]

Tom found the issue.

When I was troubleshooting when I originally made the file, I used a FTP client to verify everything was in order. This in turn created .listing files in the image folders. There was no need to delete hidden files, so FOG wasn’t programmed to do so… until now! The hidden files then meant that rmdir would fail because the folders contained files.

But thanks to Tom’s awesome work it should now work every time

[Sebastian Roth]

@Quazz At this stage of upload the client is kind of reporting back to the FOG server - triggering the image file move via FTP. Please see if you have any errors in the apache log (/var/log/apache/error.log) when you see this messages on screen!

As well you can try sending that request from a different (linux) machine to see what’s going on.

wget --post-data="mac=<mac>&type=up&osid=<osid>" -qO - http://${web}service/Post_Stage2.php

Put in <mac> and <osid> to match your client/image information…

The (empty) Error returned: message on screen seems to tell us that there was no error returned. That’s why I guess you should see something in the apache error log…

[Quazz]

@Wayne-Workman I can manually connect to the FTP by copy pasting the password from any location.

Permissions are 777 (set by chmod -R 777), ownership is fog:fog (chown -R fog:fog) and have been for a while now.

Running Ubuntu, so Selinux is not even installed at all.

I believe Tom checked those things as well when we had join.me session.

I double checked everything multiple times to make sure the issue wasn’t on my side, but I’ll explore some more out there ideas now. Will report back if I have some progress.

[Wayne Workman]

@Quazz I’ve had this error before, it turned out to be an FTP thing every time for me - but with different causes. a bad password set for the fog account, or bad permissions in the /images directory, or selinux turned on.

My suggestion would be to comb through all the FTP stuff again - but slower this time.

[Quazz]

This is what I get with latest update, not sure if this happens before the FTP move (since the images aren’t moved yet)

[Quazz]

@Wayne-Workman Yes.

I did notice that in the prep stages of the capture it sets permissions for /images/macaddress rather than /images/nameofimage ,not sure if that’s relevant or not though.

On latest trunk I get the same error as https://forums.fogproject.org/topic/7153/unable-to-upload-image-error-returned-type-2-file-var-www-fog-lib-fog-fogftp-class-php/2

Obviously, deleting the folders prior to capture each time would get tedious after a while, so I’m hoping for a better solution.

[Wayne Workman]

@Quazz Does /home/fog exist ?

[Quazz]

@Sebastian-Roth Well, it’s definitely in ext4, so I’ll try to up the FTP timeout and see if that helps.

[Sebastian Roth]

@Quazz said:

but from a quick test it did seem to be able to move the images (if the target directory didn’t already exist)

This sounds exactly like the issue I had some time ago with images being stored on a ext3 partition (see below). Possibly you are not running into the same but a similar issue - FTP timing out because deleting the old (huge?) image takes some time. Maybe play with FOG Setting FOG_FTP_TIMEOUT…

@Sebastian-Roth said in Failed to create deployment tasking…:

Learned something new … (pretty much every day!)
ext3 does some kind of defragmentation/reallocation foooo when files are being deleted!
http://www.depesz.com/2010/04/04/how-to-remove-backups/
I (re)moved all my images, re-formated with ext4, put it all back together and now everything seems fine. Deleting of 150GB image takes less than 10 seconds. Awesome!!! I really wonder why very little people have issues with this. Is ext3 not in use in most recent distributions??

[Quazz]

@Sebastian-Roth

/dev/mapper/ubuntu--vg-root on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)

I haven’t had the time to test it properly since yesterday, Tom was pretty busy yesterday as well, but from a quick test it did seem to be able to move the images (if the target directory didn’t already exist)

[Sebastian Roth]

@Quazz If you still cannot make it work show us the output of mount here in the forums. Possibly the filesystem is ext3?

[Quazz]

@Wayne-Workman Ubuntu, and yes all disabled. Don’t worry though, Tom nearly has it all sorted, should be ok tomorrow.

[Wayne Workman]

@Quazz what OS do you use? Have you checked selinux, firewall?

[Quazz]

@Tom-Elliott

root@foghost:/var/log/fog# ls -la /images/dev
totaal 16
drwxrwxrwx  4 fog  root 4096 apr 11 14:47 .
drwxrwxrwx 24 fog  root 4096 mrt 15 15:00 ..
drwxrwxrwx  2 root root 4096 apr 11 14:46 08002710d395
drwxrwxrwx  2 root root 4096 apr 11 14:48 0800276fc5ee
-rwxrwxrwx  1 fog  root    0 mrt  2 17:05 .mntcheck

So, I guess it’s not a permission issue then. Same permissions for /images

Also I found the vsftpd log, it was named xferlog (after some googling this is merely a transfer log?)

Mon Apr 11 14:46:45 2016 1 192.168.1.154 0 /images/W7PRO64FR b _ i r fog ftp 0 * i
Mon Apr 11 15:07:29 2016 1 192.168.1.154 0 /images/W7HPR64NL b _ i r fog ftp 0 * i

This is the log of the images of just now.

[Tom Elliott]

@Quazz What’s the modifiable permissions?

root:root shouldn’t matter much, but in the move process, itself, it should be set to fog:fog or fog:root. How’s it maintained in dev? Here you want to just output the listed output of the directory.

Try:

chmod -R 777 /images

It should correct the issues for you, though test would be needed to validate.

[Quazz]

@Tom-Elliott The log thing is definitely fixed, but it seems it still doesn’t move images. I noticed that it by default assigns root:root as ownership of the folders in /images/dev, could this be the problem?

[Tom Elliott]

@Quazz Hit me on chat, we should do a “join.me” session I think.

[Quazz]

@Tom-Elliott There is no vsftpd logfile! Most strange, since it’s definitely enabled in /etc/vsftp.conf

However, there is this when I check the status of the service: (I’ve been trying to upload images the past few hours)

apr 11 10:51:38 foghost systemd[1]: Started vsftpd FTP server.
apr 11 10:59:04 foghost vsftpd[10206]: pam_unix(vsftpd:auth): authentication failure; logname= uid=0 euid=0 tty=ftp ruser=fog rhost=192.168.1.154  user=fog
apr 11 11:55:12 foghost vsftpd[11194]: pam_unix(vsftpd:auth): check pass; user unknown
apr 11 11:55:12 foghost vsftpd[11194]: pam_unix(vsftpd:auth): authentication failure; logname= uid=0 euid=0 tty=ftp ruser=anonymous rhost=192.168.1.31
apr 11 11:55:32 foghost vsftpd[11196]: pam_unix(vsftpd:auth): authentication failure; logname= uid=0 euid=0 tty=ftp ruser=fog rhost=192.168.1.31  user=fog
apr 11 11:55:40 foghost vsftpd[11199]: pam_unix(vsftpd:auth): authentication failure; logname= uid=0 euid=0 tty=ftp ruser=fog rhost=192.168.1.31  user=fog
apr 11 11:56:23 foghost vsftpd[11212]: pam_unix(vsftpd:auth): check pass; user unknown
apr 11 11:56:23 foghost vsftpd[11212]: pam_unix(vsftpd:auth): authentication failure; logname= uid=0 euid=0 tty=ftp ruser=anonymous rhost=192.168.1.31
apr 11 11:56:26 foghost vsftpd[11214]: pam_unix(vsftpd:auth): authentication failure; logname= uid=0 euid=0 tty=ftp ruser=fog rhost=192.168.1.31  user=fog

So clearly it passes along the wrong password for some reason, but I can’t figure out why.