Capture task not completing after finishing and then loops

[Wayne Workman]

@arainero According to the output you provided, the problem computer’s MAC is fcaa14310ad4, and it’s not properly moving from the /dev folder to /images.

that move is done via FTP at the very end of the upload.

I can see that the image is about 1,010 GB roughly.

I think something is wrong with your FTP credentials. Those are the username and password set in your storage node here: Storage Management -> [node name] -> User and Pass

Use those credentials to FTP into the server from a remote computer. Make sure you can see what’s inside of /images and /images/dev during the FTP session. You should see a folder called /images/dev/fcaa14310ad4. Try to upload a file to the /images folder, see if that works. https://wiki.fogproject.org/wiki/index.php/Troubleshoot_FTP#Testing_FTP

What filesystem are you using?

[Wayne Workman]

and, you could just manually move the folder for temporary. We still need to get this fixed though.

mv /images/dev/fcaa14310ad4 /images/ExactImageNameGoesHere

Please still try to figure out why the move via FTP is not working.

[arainero]

@Wayne-Workman I am taking another image now since the drive was filled up by the previous one. I will update again when I attempt this. As of right now I can FTP to those folders from a different computer.

[arainero]

@Wayne-Workman Here is a video of the ending of the capture process. As you can see, it finishes and starts right back up. We turned the computer off when it got to the calculating bitmap screen.

https://www.youtube.com/watch?v=cdoXnM_DHrw&feature=youtu.be&t=1m2s

I then made a folder in /images 1282015 (the of the name of the image) and moved the files from /images/dev/fcaa14310b5b/ to /images/1282015. I am now multicasting this to another computer to see if it works.

However, that won’t fix the root problem as you said.

I did notice that the files created in /images/dev/fcaa14310b5b/ were created as user root with rw-r–r-- permissions. I was able to move the files from /images/dev/fcaa14310b5b/ to /images via ftp. I was unable to move the files from /images to /images/1282015 via ftp. I had to use mv to move them.

It may be because I made the fold 1282015 as root and it is owned by root:root.

[arainero]

It was short lived. After about 10 minutes this was displayed http://i.imgur.com/WnSPX10.jpg

[Wayne Workman]

@arainero I watched the video.

I don’t know what a “Earth Shaker” is, you can tell your co-worker.

Why are you using a RAW image type? That’s why your images are so big.

Please tell us more about the system you’re capturing from. Is it Windows? Is it Linux? What OS?

RAW image types should always be the VERY LAST choice!

[arainero]

@Wayne-Workman Earthshaker is from a video game lol.

Where do you set what the image type is? I don’t believe I have ever seen a RAW option.

I am capturing Windows 7 machines with two hard drives.

[Wayne Workman]

@arainero said:

https://www.youtube.com/watch?v=cdoXnM_DHrw&feature=youtu.be&t=1m2s

It’s right there in your video, it says “File system: RAW”

For a example of a non-raw, see this: https://www.youtube.com/watch?v=VZXBCdULUbk
Of course, that is a troubleshooting video I uploaded for another issue - but it does show “File system: ntfs” in the video as an example.

Can you give a screenshot of your image’s settings?

Here’s what I’d choose for a machine with multiple disks:

[arainero]

@Wayne-Workman I believe that youtube link is the one I supplied.

Here is a picture of the image settings screen. I used to use the default setting of 6 for compression, but now I have been experimenting when this problem started happening.

[Wayne Workman]

@arainero said:

@Wayne-Workman I believe that youtube link is the one I supplied.

fixed.

Is the 1TB drive encrypted at a hardware level? bitlocker?

[arainero]

@Wayne-Workman There is no encryption happening.

[Tom Elliott]

Is this supposed to be multipartition all disk or are you only trying to do a single disk?

[arainero]

@Tom-Elliott This is a Windows 7 image. There are two disks, the first one has the Win7 OS and the second one is the 1TB storage disk. So multipartition, all disk.

I have used these options before and never had a problem with a capture. This problem has only come to surface after my multicasting problem was fixed, from https://forums.fogproject.org/topic/6129/multicasting-stuck-on-starting-to-restore-image

In regards to my previous thread, I did change password for the FTP, FOG user, and FOG config to trouble shoot that problem before. I did change them back though. I am not sure if that would be related to this new problem or not as I can FTP in just fine.

[arainero]

@Tom-Elliott Do you think I should try updating to the latest trunk version or hold off on that for the time being?

[Tom Elliott]

I say hold off, updating won’t hurt or help anything (I don’t think).

[arainero]

@Tom-Elliott Is there anything that I should test or attempt to narrow down what may be causing this?

[Sebastian Roth]

@arainero Re-watching the video you posted three days ago I noticed something really strange. We see the end of one partclone cycle which is capturing from /dev/sdb. Notice that there is something missing. There is no number showing the partition in that first run. Then we see a black screen at 1:14 saying:

* Image uploaded
* Processing Partitions: /dev/sdb1 (1)
* Using parclone.ntfs

I am still not sure what’s going on but I feel certain that this is really not what we expect to see (first capture full sdb in RAW and then sdb1 as NTFS). Please do me a favour and boot this client into debug mode (Host -> Basic Tasks -> Debug). I am not exactly sure what we are looking for so we might need several iterations on this.
First try the command fogpartinfo --list-parts /dev/sdb Depending on the version you are at this might fail with “command not found”. That’s ok then. After that please run lsblk -pno KNAME,MAJ:MIN -x KNAME | grep sdb and grep sdb /proc/partitions. Please take a picture and post this here so we can see as well.

[Wayne Workman]

@Sebastian-Roth Good catch.

PartClone not only says raw, but the partition number is missing! A big clue indeed.

I can confirm that the partition number should be there, if you look at the example video below the partition number is included in the PartClone screen.

When would the upload process choose RAW when the image type is not set as RAW?

[arainero]

@Sebastian-Roth Here is the picture.

This includes output of requested sdb and I included sda just in case that proved useful too.

http://i.imgur.com/jnhNKhW.jpg

[Sebastian Roth]

Checking our script code again and again I found out what’s causing this!! Lately we re-worked the code on enumerating the partitions on disk(s). In earlier revisions a self written tool called fogpartinfo was used. This has caused some trouble (for example on brand new empty hard disks). So we decided to get rid of it and use well known linux commands to get the partition information. This is a lot more flexible but we are still in the process of getting it fail proof in all cases. But turns out that we haven’t been aware of “Multiple Partition Image - All Disks” being a bit more tricky.
Please download init.xz/init_32.xz from here https://drive.google.com/folderview?id=0B-bOeHjoUmyMazJLZDhGaEl5VTQ&usp=sharing and put into place in /var/www/fog/service/ipxe/ or /var/www/html/fog/service/ipxe/ (probably a good idea to rename the original files instead of just overwriting them!)

Please test and report back if capturing is working properly again.

@Wayne-Workman said:

When would the upload process choose RAW when the image type is not set as RAW?

The scripts collect filesystem information to be able to run the correct partclone command which corresponds/is aware of the filesystem on each particular partition. sdb has no filesystem and is cloned RAW because of that.