Not NTFS Partition error 34 during capture

[John Sartoris]

Server

• FOG Version: 1.4.4
• OS: Ubuntu 14.04.5 LTS

Description

I’m trying to capture my second Windows 10 Creators 1703 image and I’m running into different problems than last time. I’ve experienced the shutdown tweak to sysprep with my previous image. I still think that is strange that sysprep can’t shutdown directly.

So my current problem, I forked an imaged from my previous working image so that I could add approx 100gb of extra CAD software. Then I went through the same sysprep process that worked before and with the forked image I am getting an error from Partclone, “ntfsclone-ng.c: NOT NTFS partition, ntfs mount error 34”.

I’ve gone back to my my previous image from before the fork to test the fog server and this “base” image still uploads successfully. After that it drops back to console and says “ensure disk space is good to go”. Server has 175gb free, image has 108gb used. Even uncompressed I should have plenty of space.

On subsequent reboots it reports with a different message. I figure this is because the system is in a broken state following the failed upload.

[Quazz]

For your second one, I recommend running chkdsk /f . Your MFT seems to have some issues there.

Exit code 1 more or less means partclone exited earlier than expected, the suggestion is somewhat misleading. This could be caused by any number of things.

Are you using resizable image type?

[John Sartoris]

Forgot to mention that I tried running “chkdsk /f”. Reboot, checks, doesn’t report errors, still has same problem with the backup.

Operating System = Windows 10 - (9)
Image Type = Single Disk - Resizable - (1)
Partition = Everything - (1)
Comp = 21
protect = no
enabled = yes
replicate = yes
Image Manager = Partclone Gzip

If I assign my “base” vm to this image and capture it does so successfully. So I am not expecting this to be a fog server related issue.

[Quazz]

@john-sartoris I’m not entirely sure about this, but Gzip has a max compression level of 9 I believe.

Higher values can be entered because ZSTD can go higher.

Perhaps something to do with that?

[John Sartoris]

@quazz

Like I said though, if I use a different VM to capture/backup into this “image” definition, it is successful. I can try lowering the compression to 9 and try again. I also started this image testing out the new ZSTD compression and it did not work so I switch to setting that worked (max compression, gzip). (Very impressed by the numbers, hoped it would work)

[Sebastian Roth]

@John-Sartoris Please make sure windows shuts down properly (energy settings -> untick/disable the “Turn on fast startup…” foo! See here.

As well you might look into this wiki article: https://wiki.fogproject.org/wiki/index.php?title=Windows_Dirty_Bit

[John Sartoris]

@sebastian-roth
Tried it, and failed. I was hopeful. The box was checked even thought hibernate was off, and the extra software specifically asked to disable it…

[Sebastian Roth]

@john-sartoris said in Not NTFS Partition error 34 during capture:

Tried it, and failed.

What exactly did you try? Do you still get the exact same error (“Run lists overlap…”)?

Maybe try scheduling a debug upload task and when you get to the terminal run fixparts /dev/sda and answer y(es), w(rite), y(es). After that start the upload process by issuing fog command. Please see if things changed.

In case you run into errors please post another picture so we can see.

[John Sartoris]

@sebastian-roth
I was trying your suggestion about “fast startup”. I had skip double checking that option, as I said because disabling hibernate is supposed to disable “fast startup”, and that the extra cad software specifically asks to disable it.

Yes, I still get the run list overlap, but that is following the not ntfs partition and reboot, the system pxe boots back to fog and doesn’t start uploading at all just straight to “run list overlap”.

As for my my process and error messages regarding this image, I’ll try to line that out here. This is going to start back from the previous image. I build my images in vmware on an ESXi server and take heavy advantage of snapshots to rollback to useful points.

1.  (July 2016) finish windows updates
2.  (July 2016) snapshot point 1
3.  (July 2016) delete active install user profile
4.  (July 2016) run windows disk cleanup
5.  (July 2016) run sysprep script - run as admin
	5a.(script)  purge temp files
	5b.(script)  stop fog service
	5c.(script)  purge log files
	5d.(script)  purge cleanup cache files
	5e.(script)  remove default user profile to force use of network default
	5f.(script)  clear wsus ids from registry
	5g.(script)  clear hklm\system\setup\upgrade key for tracking upgrade history
	5h.(script)  clear regkey of any autologon remnants 
	5i.(script)  revert some regkeys to default from Group Policy settings
	5j.(script)  rearm office 2016 install (fix duplicate keys in KMS and WSUS)
	5k.(script)  start "sysprep /generalize /oobe /quit /unattend:unattend.xml"
6. (July 2016) run "shutdown -s -t 0 -f" manually
7. (July 2016) capture fog image(upload successfully as 2016Base-R2.0)
8. (July 2016) rollback to snapshot point 1
9. (July 2016) clone VM fork to PLTW image
10. (July 2016) install extra cad software
11.  (July 2016) snapshot point PLTW2
12.  (July 2016) delete active install user profile
13. (July 2016) repeat steps 3-6
14. (July 2016) capture fog image(upload successfully as 2016PLTW-R2.0)

Jump forward to Dec 2016

Base Update - R2.1

15. rollback base to snapshot point 1
16. update avast av to version 1609
17. update java
18. update other base software
19. disable windows "DiagTrack" service
20. cleanup other reg keys for updated policy settings
21. snapshot point Base2
22. repeat steps 3-6
23. capture fog image(upload successfully as 2016Base-R2.1)

Jump to (Now) July 2017

Base Update - R2.2

24. roleback to Base2
25. windows updates
26. windows upgrade via Windows update to 1703
27. update base software
28. update Adobe Suite to 2017 versions
29. install HP HBMA driver (virtual mac address for docks)
30. snapshot Base3
31. repeat steps 3-6
32. capture fog image(upload successfully as 2017Base-R2.2)

PLTW update - R2.2 Skip R2.1 to match base version

33. roll back to snapshot point PLTW2
34. update avast av to version 1609
35. windows updates
36. windows upgrade via Windows update to 1703
37. windows updates
38. update cad software
39. update base software
40. snapshot PLTW3
41. repeat steps 3-6
42. capture fog image(upload fails as 2017PLTW-R2.2)
	42a. partclone uploads sda1 - no error
	42b. partclone errors sda2 - not ntfs partition
	42c. partclone exits to console - error "disk space is good to go"
	42d. computer reboots in 1 minute
	42e. pxe boot - fog picks up the process again
	42f. fog errors with "run list" before starting partclone.

[Sebastian Roth]

@John-Sartoris What about the fixparts command I mentioned? Have you tried it? Got an error? Which error?

[John Sartoris]

@sebastian-roth said in Not NTFS Partition error 34 during capture:

Maybe try scheduling a debug upload task and when you get to the terminal run fixparts /dev/sda2 and answer y(es), w(rite), y(es). After that start the upload process by issuing fog command. Please see if things changed.

I didn’t get time yesterday to focus on this. Here’s the output I received.

I’m not sure this procedure is correct. /dev/sda2 being a partition shouldn’t have a partition table…

I rolled back to the snapshot and ran again with “fixparts /dev/sda” and received no warnings, did a write anyway. Reran the “fog” the process again even though there were no changes. I watched closer all the lines that go by, and notices a few of interest.

It finds and identifies 2 NTFS partitions. Ntfs resize happens without error. Problem only appears with the actual partclone capture.

Clearing the ntfs flag … Failed???

This is just before fog saves the new partition layout and starts the partclone captures.

[John Sartoris]

If I change the image settings to “Multiple partition image - single disk (not resizable) -(2)”, the upload proceeds. This tells me the problem is with the partition resize.

This might present me a problem. Previous years we had 500g-1tb spinning drives, this year we switched to 250g ssd drives. The reported device size in Partclone is 267.9g.

I swapped back to zstd compression and 200m split. The capture failed out with this. Trying again with gzip.

edit: Missed the image

[Tom Elliott]

@john-sartoris Way too many pictures with minimal text inbetween makes it hard to follow.

That all said.

/dev/sda2 does not have a partition table.

FOG’s capture system works as such:

Get the current layout.
Attempt to shrink partitions as possible/required. (All partitions)
Once all partitions are shrunk, get the partition table of the disk.
The problem seems to be something with /dev/sda2 though we don’t know what. Something is causing partclone to not even be able to mount /dev/sda2.

[John Sartoris]

@tom-elliott

@tom-elliott said in Not NTFS Partition error 34 during capture:

@john-sartoris Way too many pictures with minimal text inbetween makes it hard to follow.

Sorry about that. I figured that would be the best way to capture the sequence of output from fog. The actual problem isn’t giving much information. Hopefully something else inline would stand out to someone with more detailed experience.

[Sebastian Roth]

@John-Sartoris Maybe provide more information on the partition layout. See in d1.partitions and d1.minimum.partitions in the image directory on your server.

[John Sartoris]

@sebastian-roth said in Not NTFS Partition error 34 during capture:

@John-Sartoris Maybe provide more information on the partition layout. See in d1.partitions and d1.minimum.partitions in the image directory on your server.

d1.minimum.partitions is not in the /images/dev/082… upload folder

d1.partitions

label: dos
label-id: 0x86308630
device: /dev/sda
unit: sectors

/dev/sda1 : start=        2048, size=     1024000, type=7, bootable
/dev/sda2 : start=     1026048, size=   523259904, type=7

[Sebastian Roth]

@john-sartoris said in Not NTFS Partition error 34 during capture:

d1.minimum.partitions is not in the /images/dev/082… upload folder

This is because the image is set to non-resizable right now. Nevertheless the partition layout is pretty straight forward and I can’t think of why this wouldn’t work with resizable. At least when looking at the partitions. Possibly there is something going wrong when FOG does shrink the partitions before capturing those. You might want to try ntfsfix -b -d /dev/sda2 in another debug upload session. This is what FOG is doing when it says “Clearing ntfs flag”.

[Quazz]

It’s reporting 2 NTFS partitions. Is this correct with your partition layout?

A default Windows installation will only have one.

[John Sartoris]

@quazz said in Not NTFS Partition error 34 during capture:

It’s reporting 2 NTFS partitions. Is this correct with your partition layout?

A default Windows installation will only have one.

If you manually create a full disk partition during the installer, 1 is correct. If you install to a blank disk, the installer will create 2. The first being 100-500mb depending on version. It has been this way since Win7, I think. This small partition is the “Active” boot partition and to the best of my knowledge is used for booting bitlocker encrypted partitions. The primary boot files and some system level things need to remain unencrypted. Fog has specific rules to leave these small partitions basically untouched.

[Quazz]

@john-sartoris That partition should be FAT32.