Error 'Could not open inode 'XXXXXX" through the library'

[Jarl2.0]

FOG Verision: 1.4.4
Server: Ubuntu 16.04
Client Windows 10 (1709 Fall update)
Image: Single Disk - Re sizable, Everything, Partclone Zstd Compression 12

I’m getting the attached error when trying to take an image from a client after the Windows version 1709 update. I’ve used this model of PC to take images in the past before so I don’t think its a hardware issue.

https://imgur.com/a/vrhGk

[Sebastian Roth]

@Jarl2-0 Please make sure you have fast boot disabled before trying to capture that image. As well read through this wiki article: https://wiki.fogproject.org/wiki/index.php?title=Windows_Dirty_Bit

[Jarl2.0]

I’ve confirmed that fast boot is disabled and I am still getting the error.

[Sebastian Roth]

@Jarl2-0 We have had the issue posted in the forums a couple of times lately. Unfortunately we are not able to pin it down exactly. Possibly M$ has changed something with 1709 but on the other hand not all users see this issue. Quite often it can be fixed by disabling fast boot - which should be disabled on all machines anyway!

I’d advise you to run a full chkdsk on Windows, maybe even defrag the disk and run chkdsk again. But this is just wild guessing.

The Input/output error at the end of the message to me sounds like the filesystem inode is pointing to a sector that is beyond the disk. But I have no idea really.

[Jarl2.0]

@sebastian-roth Tried your suggestions still having no luck, I’ll look at some of the other posts on the forums and see if I can find anything that works.

[Jarl2.0]

Went through and imaged a new laptop with a 1703 image was able to make some changes and take an image from 1703 just fine. Upgraded to 1709 and got the same inode error, something must have change I’m going to try to sysprep the image to see if that helps but sysprep has been failing for me on 1709 as well.

[Jarl2.0]

I have some more info on this, it seems to only happen for me after I remove the old Windows versions from a PC I want to take an image of. A freshly updated PC to 1709 will have no issues until you try to cleanup those files. Which is annoying because they take up about 30gigs of space.

[Sebastian Roth]

@Jarl2-0 Thanks for updating this. Are you able to reproduce this issue? Like clean install, do all upgrades to the latest, test capture. Then do a cleanup and test capture again. Best if it could be done in a VM environment where you can use snapshots to go back to an older state.

How is the cleanup done? Through the normal Windows disk cleanup or using third party tools?

[Jarl2.0]

@sebastian-roth Yes I’m able to reproduce this. Clean install upgrade to 1709 test capture > Windows cleanup to remove old patch data and previous versions of the OS.

The cleanup is done with windows disk cleanup.

Another update is that I am able to take an update with the “Multiple Partition Image - Single Disk” image type. I would prefer Single Disk Re-sizable but it always failed.

These settings are working for us to capture 1709 images.

[Sebastian Roth]

@Jarl2-0 Gosh, NTFS is a complex topic. I’ve tried reading up on this and figuring out what goes wrong in the code. No answers yet. But I see that we have more than enough reports by other users as well and you seem to kind of nail it down. So I hope we can figure this out together. I just sent an e-mail to tuxera.com as we use their latest ntfs-3g/ntfsprogs code compiled in the buildroot environment that we use for FOS. Keeping my fingers crossed that we get an answer from them.

Yes I’m able to reproduce this. Clean install upgrade to 1709 test capture > Windows cleanup to remove old patch data and previous versions of the OS. The cleanup is done with windows disk cleanup.

Can you please be more specific on how exactly you reproduced the issue? Does it happen on the clean install or after updating to 1709 or only after running the disk cleanup tool? How many times did you test this? Please don’t get me wrong, I am not saying this is not right. I just want to nail it down as much as possible to we can give tuxera people good instructions on how to reproduce this so they can go ahead and find/fix the issue in the code.

[Jarl2.0]

@sebastian-roth

It was after I removed the previous Windows version is when I had issues. I could take an image after upgrading to 1709 but when I went to clean out the previous windows version the next attempt to take an image would fail. I would say this happened about 4-5 times with different PCs (same model) before I pin pointed what step caused the image to fail.

Steps for me were like this.

1. Win10 machine with 1703 - Image Success
2. Upgrade to 1709 - Image Success
3. Remove old patch data - Image Success
4. Remove previous Windows version - Image failed

I removed the patch data and previous Windows install via Disk cleanup > Cleanup system files.

[Sebastian Roth]

@Jarl2-0 Thanks for the details. Sounds pretty straight forward. Though I still can’t imagine what exactly would break for ntfsresize. Unfortunately I haven’t got an answer from tucera yet.

Is there anyone else out there who is able to replicate the issue as described here? @Testers @Moderators

[Quazz]

@sebastian-roth I have had the same issue on a Windows 7 installation (had been in use for maybe a year) which we had to clone as it was.

Non-resizable worked fine, resizable failed with the inode message.

edit: Sorry I can’t be more helpful than that, it was the only instance I could recall and of course I am not sure what they did on those computers.

edit2: It could have been hardware related, they had been in use for a while, after all. (although SMART certainly didn’t complain). And of course did chkdsk /r and such with no change. (and no errors found)

[Sebastian Roth]

@Jarl2-0 We got an answer! Someone is interested to look into this, analyse metadata and try to fix the issue. But he also suggested to defrag the disk before doing the resize (capture)! Could be a workaround till we figure this out. Please give it a try and report back.

[Sebastian Roth]

@Jarl2-0 I thought I’d set up a test machine to replicate the issue but I am very short on time at the moment and therefore would ask you to create a metadata dump to send to the developer to look into. Quoting from his mail:

I am interested in really fixing the issue, but I need to have a full picture of the metadata. This will not contain any user data, though the file names will be visible. Nevertheless this will be a big file which you will have to upload to a server for me to download (it will not fix in an email attachment).
To extract the metadata image, do (as root) :
ntfsclone -mst -O - /dev/xxx | gzip > metadata.gz (replace xxx by the partition name, like sda2)
If you have a powerful computer, you will get better compression by using “xz -T0” (without the quotes) instead of gzip.
This will require the partition to be consistent, you may have to do a chkdsk on Windows beforehand. If you cannot, you may add the extra option --ignore-fs-check but I might not be able to reproduce the issue.

Please let me know if you need space to upload that metadata file somewhere but I guess you have google drive or dropbox or something like that. Please send me the link (in a private chat message if you like) and I will forward it to the developer. Thanks in advance.

[rj7658]

I am having the same issue as described here. I was running FOG version 1.4.0 first few tries. I upgraded to version 1.5.0 to see if it helped. I tried check disk and defrag using built in windows tools. I did manually resizing my image using Disk Management and Windows couldn’t complete the resize if I drop free space lower than 30%. The Image size of Windows is 37.9G and couldn’t shrink below 58.4G. I dont like the idea of using a non-resizable disk because it adds the extra step of extending the volume manually, BUT it does work.

If I had to guess, Microsoft is writing something out there that FOG can’t move and is in conflict with the leaving only 2G of free space.

[george1421]

@rj7658 Another solution (which we used before fog supported resizable disks) where to create the reference image in a VM with the disk smaller than the smallest size we had in our environment. So at the time we created our reference image on a 40GB virtual disk. Captured and deployed with FOG, then in the setupcomplete.cmd piped some commands into diskpart to expand the disk to the size of the target hard drive.

You could use a similar concept by creating your reference image on an artificially small hard drive, capture with FOG single disk resizable, then when FOG deploys to a target system it will expand to the size of the target hard drive.

[Sebastian Roth]

@rj7658 Thanks for bringing the issue up again. As you see in my last post I had contact to one of the developers already and he offered to look into it. Would you be able to setup a test machine to try and replicate the issue and provide debug data as described in my last post?

[Sebastian Roth]

@rj7658 @george1421 @Jarl2-0 Ok, got some news here. Contacting the developer about this again and he told me that others have reported an issue as well and provided debug meta data to look at.

Here are the two commits kind of related to that issue:
https://sourceforge.net/p/ntfs-3g/ntfs-3g/ci/f334c1fdc397d92d2bb4aa9cf75b75dbb33eff40/
and
https://sourceforge.net/p/ntfs-3g/ntfs-3g/ci/1f8b751341944545eceb6037176f288843eb7b26/

@Tom-Elliott Any chance we could add that new version / or the patches to our FOS image without too much hassle? Or should we just wait till the fix is upstream? I really wonder that we have seen those errors a couple of times months ago and now it is all quiet again.

Hint for everyone: Defragging the disk might help even without the patches.

[Tom Elliott]

@sebastian-roth I’ve made a patch that includes these fixes. I’m currently building the new init’s which will take quite a long time, but I have uploaded the patch to our fos repository. Sorry it will take a while to get some inits built.