"Failed to create deployment tasking..."
-
[quote=“Uncle Frank, post: 42228, member: 28116”]Ok I get what you’re saying about OSID 1/2 check and moving but I still wonder why ‘$ftp->delete($dest)’ is called before moving the new image to that destination. If things go wrong with moving you end up with no image in ‘/images’. What about renaming ‘/images/<imagePathName>’ e.g. to ‘/images/<imagePathName><random seed>’ and deleting it after moving from dev was successful??[/quote]
I thought i would add, the only time i’ve really seen anything “go wrong” with moving the image to /images is when there’s an issue with ftp working. In which case, the delete operation will have failed too.
-
[quote=“Uncle Frank, post: 42829, member: 28116”]dump[/quote]
I am dump… sorry for this. I was in a hurry today and just remembered that this “issue” is still not solved from my point of view. I quite often see newly uploaded images not being moved BUT the old image being deleted! I am getting used to it but it’s definitely a bit scary if you encounter this at first.
Probably a problem with how my vsftpd is setup?? So far I didn’t have the time to look into this any further…
Depending on the scenario you either don’t need extra space (FOG is on the same server as storage node is!) or you do need a lot of it (storage is a different node). Would it be a good idea to check if FOG and storage are the same server, move things locally in the filesystem and delete only if rename went well?? What do you think?
Maybe it’s just me but I think I’ve read about other people also wondering about “missing uploaded images”. -
I can’t put a finger on when the image went missing… that was a while back. I’ve simply shrugged, rebuilt a new image, WRITE PROTECTED all the images, and moved on.
But, my image DID go missing that one time, and I didn’t know what the hell happened.
-
It’s been a long time since I looked into this but I didn’t have enough time until now…
I am now able to reproduce this on our machines and then captured the FTP traffic to see exactly what’s happening when things go wrong.
[LIST]
[]Case 1: Create a complete new image in the web GUI and upload. Works like a charm!
[]Case 2: Upload a second (third …) time. Bam! Image does not get moved…
[*]Case 3: Delete /image/<name> before uploading. And again, rename works like a charm…
[/LIST]
The problem is that our image is very big (150GB+). But also smaller (50GB) images fail on out servers. [B]FTP timeout is set to 10 seconds [/B]but deleting such hugh files takes longer than 10 seconds on our system…I cranked it up to 90 seconds (PHP default value) and the 50GB images work now. 150GB image is still an issue (testing…). Maybe it’s just our storage system that takes ages to delete big files…??
-
Did you experience an image going missing?
-
Nope, never ever so far!
-
If added two new capabilities to the general settings of the fog configuration. FOG_FTP_TIMEOUT and FOG_FTP_PORT. The default timeout is now set to 90 but should reflect whatever is put in this field though I still need to add some error checking.
-
Thanks Tom! That’s great! Although I think we have a real issue with our virtualization software (proxmox). Still trying to find out why deleting big files takes MINUTES… definitely shouldn’t be the case!!
-
Learned something new … (pretty much every day!)
ext3 does some kind of defragmentation/reallocation foooo when files are being deleted!
[QUOTE]ext3, when freeing disk space is doing some kind of defragmentation, which moves some data around to maximize continuous free disk space.
So, when we had 3 large files (blocks A, B and C), and some free space (.):
AAAAAAAABBBBBBBBCCCCCCCC…
and I removed whole A:
…BBBBBBBBCCCCCCCC…
ext3 relocated some blocks (most likely from C):
CCCCCCCCBBBBBBBB…[/QUOTE]
[url]http://www.depesz.com/2010/04/04/how-to-remove-backups/[/url]I (re)moved all my images, re-formated with ext4, put it all back together and now everything seems fine. Deleting of 150GB image takes less than 10 seconds. Awesome!!! I really wonder why very little people have issues with this. Is ext3 not in use in most recent distributions??
[B]Could anyone please add [SOLVED] to the title, thanks![/B]
-
So the moral of the story is to use ext4 ?
-
[quote=“Wayne Workman, post: 45339, member: 28155”]So the moral of the story is to use ext4 ?[/quote]
Well, there are a lot of other nice filesystems out there too! ext4 is working for us right now and I am sure it will for others having an issue with deleting hugh files on ext3. But I am not saying that ext4 is the only fs that can do this…
