Strange Behavior when Uploading Image

[mecsr]

I’ve been working on understanding FOG better since our FOG expert left here for better places, and it’s been going alright - a bit of a bumpy road, but not too bad. However, last night I tried to upload my first image (not the first for our FOG server, but for me, yes) using a Virtual Machine. It was going a bit slower than I expected, but seemed to be working.

And then I came in this morning and saw this:

I also noticed that I couldn’t log in to the FOG server for management or anything else. It appears that either this or something else turned off our server. I was able to turn it back on, but upon logging back in to the web management GUI, I get the following message:

I would appreciate some insight into what kind of course of action I should take, and what I can do to perhaps prevent this from happening again. Thank you!

[JJ Fullmer]

@mecsr What happens when you try to click install/upgrade?
That process won’t delete anything or anything of that sort, but that part of the install is now automated.

Also you should update to the latest trunk to get the latest kernel, that could help.

Hopefully some of the smarter people on here like @Tom-Elliott @Sebastian-Roth @Wayne-Workman @george1421 will have some idea on the cpu stall business

[JJ Fullmer]

Also for reference, since that server is one I set up when I worked there I know some things.

It’s on Ubuntu server 15.04 64 bit
As I recall it has a sandy bridge quad core intel i5
16 GB RAM
And it’s a HP 6300 Pro workstation being used as a server.
It has 2 4TB drives that are synced together nightly, so unless @mecsr turned off the cronjobs it is backed up database and all if an os reinstall is needed, but hopefully that can be avoided.

[mecsr]

@Arrowhead-IT Install/upgrade actually just ended up telling me that an upgrade wasn’t needed, so that’s good. I now have access to Web management. I updated FOG two days ago, so I think I should be good in that respect.

[Wayne Workman]

To me, looks like you ran out of space.

[mecsr]

@Wayne-Workman That sounds like it might be the case. The VM for the image I was trying to upload is now giving me this message, which seems to be in line with what you’re saying:

[Wayne Workman]

@mecsr on your fog server, run df -h to see the space used/available for each partition on your server. Feel free to post the results here.

[mecsr]

@Wayne-Workman Ok, here’s what I’ve got:

Filesystem      Size  Used Avail Use% Mounted on
udev            7.8G     0  7.8G   0% /dev
tmpfs           1.6G  9.4M  1.6G   1% /run
/dev/sda2       3.6T  3.0T  431G  88% /
tmpfs           7.8G     0  7.8G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/sdb1       3.6T  2.9T  566G  84% /mnt/mist
/dev/sda1       511M  3.4M  508M   1% /boot/efi
tmpfs           1.6G     0  1.6G   0% /run/user/1000

Mod edited to use code box.

[Wayne Workman]

@mecsr Can you run find / | grep /dev/.mntcheck please to locate where your image directory is?

Also, how big is the image you’re trying to capture, In terms of space used?

[george1421]

Either way it looks like there is plenty of space of sda and sdb, unless he is uploading one whopper of an image.

I’ve seen the ntfs volume inconsistent when the target is shutdown incorrectly. Booting it back up and then properly closes the disk. I’ve also seen this with a bad / corrupt disk. Either way I’m leaning towards the target’s hard drive at fault.

[JJ Fullmer]

@mecsr Have you run test disk on the vm your uploading that image from that gave you that message?

Still unsure why uploading an image would cause a cpu halt

[JJ Fullmer]

@mecsr said:

I updated FOG two days ago, so I think I should be good in that respect.

Just fyi, there’s typically at least a few if not a bunch or very many updates to the fog trunk version any given day containing many bug fixes and optimizations. It’s a program that just keeps getting better and you get the benefits of instantly.

The current version is 6303 and you’re on 6271. So I would suggest considering updating

[Wayne Workman]

@george1421 said:

unless he is uploading one whopper of an image.

I remember once a guy uploaded a 1TB image… RAW.

[Sebastian Roth]

Not sure if this is related?!?! https://forums.fogproject.org/topic/6664/virtualbox-image-upload-r4792

Might current kernels really have issues in VMs??

[Tom Elliott]

@mecsr what’s the output of df -i?

[mecsr]

I haven’t had too much time to look at this since yesterday morning, so I haven’t really had an opportunity to run any of the suggested commands on the server yet - I will get around to that very soon. But I do think that there is an update on the situation - last night I started a computer imaging with one of the images that we already have had uploaded for quite some time. It’s worked before, but this time, on imaging, the image almost completes and then fails at the last moment. I have yet to see what the error message says.

I’m starting to to think that the suggestion of hard drive issues might be correct.

[mecsr]

@Tom-Elliott Here’s the output of df -i

Filesystem        Inodes   IUsed     IFree IUse% Mounted on
udev             2035414     522   2034892    1% /dev
tmpfs            2038213     691   2037522    1% /run
/dev/sda2      243122176 9464955 233657221    4% /
tmpfs            2038213       1   2038212    1% /dev/shm
tmpfs            2038213       7   2038206    1% /run/lock
tmpfs            2038213      15   2038198    1% /sys/fs/cgroup
/dev/sdb1      244195328 9354922 234840406    4% /mnt/mist
/dev/sda1              0       0         0     - /boot/efi
tmpfs            2038213       4   2038209    1% /run/user/1000

[mecsr]

@Wayne-Workman said:

@mecsr Can you run find / | grep /dev/.mntcheck please to locate where your image directory is?

Also, how big is the image you’re trying to capture, In terms of space used?

The image I’m trying to capture is at most, 300 GB. My image directory is just at /images.

[Wayne Workman]

@mecsr said:

@Wayne-Workman said:

@mecsr Can you run find / | grep /dev/.mntcheck please to locate where your image directory is?

Also, how big is the image you’re trying to capture, In terms of space used?

The image I’m trying to capture is at most, 300 GB. My image directory is just at /images.

So you only have 431GB of space available on / and if your image is 300GB, then the FTP move at the end may cause issues if the data is copied and then deleted, instead of just moved. Going to ask the @Developers for clarification on the process - this has come up before.

[Sebastian Roth]

FTP move should not be an issue or we should see a somehow FTP related error message - which would happen much later anyway. To me it seams like the image/VM might be corrupted. Although disk space and inodes are not running out yet.

@mecsr Have you tried running chkdsk /f within the client system as suggested on the screenshot??

I really don’t hope that this is the case but could it be the server disk is failing??