The future of partclone and therefore FOG as it is

[george1421]

@Junkhacker

Ok first of all I updated the init on the google drive (the end result here will be to provide the developers a diff file between the changes and the original).

So your deployment worked and mine did not. So we need to find out what’s different.

In my case it was a uefi image deployed to a Dell 6430 in uefi mode. Using the original inits this system deployed ok. With the partclone v0.3 it did not.

The deployed image was Win10 Ent 1803 partclone gzip compressed. OS=Win10 (9) single disk resizable all partitions, compression 6.

I’ve got tied up today so I haven’t had a chance to build a new golden image to attempt to capture and deploy with the same (upgrded) partclone.

[Junkhacker]

@george1421 mine was a bios image deployed in legacy mode. the image was a Win 7 Ent zstd compressed resizable, compression 11.
i also tested a windows 10 image that was otherwise the same.

with uefi, the UUID information not applying error i got would be important. did yours throw a visible error? do your partitions have the data intact on them, or are they corrupt? (i had problems in testing with corrupted deployment when using the --ignore_crc flag is why i ask.)

[george1421]

@Junkhacker They were corrupt. I could see the partition types as uefi boot and such, but could not mount them.

I think I have something else going on here. I just tried to recapture a uefi golden image and I have an error about getPartitionLabel: command not found on line 486, then partclone exited with error code 139. So I’m going to have to do a bit more research, maybe uefi mode is causing something if bios mode worked for you.

I may need to start with something simple like win10 bios mode and see where it starts to fall down.

[Junkhacker]

@george1421 is it possible that you tried to deploy a legacy image with partclone 0.3 and the --ignore_crc flag set, then captured it again, or something like that? or maybe captured without checksums with 0.3 and redeployed it with --ignore_crc? i know for certain that the second of those two will result in corrupt partition without reporting any problem. (it tries to skip the blocks in the image file where the checksum would be, but there it skips actual data because there’s no checksum)

[Junkhacker]

downloaded the init_p3 file and tried again. got the UUID error again, but it boots fine.

[george1421]

@Junkhacker What I’m currently testing is an legacy image captured with the original inits and then deploying with the test inits from this thread. In that case the test inits (partclone 0.3.12) should have all of the --ignore_crc switches removed.

I’m doing this to confirm we won’t have issues with previously captured images and the new inits.

Well after deploying a simple win7 bios mode its also failed to this same test computer. So at this time I need to start back from a known good state and change only one thing at a time.

[Junkhacker]

@george1421 that’s the same as i’m doing. i just thought maybe in your testing an upload might have taken place with 0.3.12 by accident. i’ve been deploying images using my dev server, some of which i copied over from my production server. i haven’t seen any problems other than the UUID thing i posted.

[george1421]

@Junkhacker Well this is all good information. In your environment (except for the UUID issue) its deploying correctly. The other thing (thinking about all of the variables here) is that I’m running 1.5.4 on my production server. I’m pretty sure that 1.5.4 inits were created with a previous release of buildroot than what I’m building against. Its possible that buildroot updated applications too, but if everything appears to work in your environment this I can take one step to rule out buildroot differences causing this issue. As well it could be this old laptop that has issues. I’ll keep working on eliminating non-issues.

Thank you for your feedback.

[Sebastian Roth]

@george1421 Great stuff you are working on this full on!! I try to follow up on what you post and test. Though I don’t have the time to engage in this at the moment.

It’s interesting you and Junkhacker seem to get different results and it would be very important to figure out why. I can’t imagine it being a UEFI vs legacy difference issue as partclone should just be deploying the actual contents of the partitions no matter what the boot type or partition layout looks like. But what do I know…

@george1421 said:

I just tried to recapture a uefi golden image and I have an error about getPartitionLabel: command not found on line 486, then partclone exited with error code 139.

Not sure which version you used because we removed getPartitionLabel stuff recently. Please make sure you use the very latest version (master branch).

@Junkhacker The UUID error you see stems from an issue we had in the inits when I created the one you use for testing. This was fixed some weeks ago.

i tried the init you supplied, and i got a "failed to set disk guid (sgdisk -U) (restoreUUIDInformation)

Can you please post a picture of that?

[Junkhacker]

@george1421 more info for you, my dev server is 1.5.5 my prod is 1.4.4. not sure what versions the images files would have all been captured on, but i’ve tested 1.4.4 or earlier (not sure) though captures with 1.5.5.

[Junkhacker]

@Sebastian-Roth

[Tom Elliott]

That looks like an error code of some sort, weird.

[Junkhacker]

@Tom-Elliott yeah, definitely doesn’t look like a UUID (sorry the image is shrank and funny, was trying to get the image size down to upload)

[Quazz]

@Junkhacker The following commit changed UUID stuff https://github.com/FOGProject/fos/commit/f49b0f7d5b0c90866a6dcdbbd5d59e529857242e

On MBR, the label-id is in the form of what appears to be an error code (0xrandomnumbersandletters) hence the output that you saw.

I don’t know for sure why it would fail, but something in that direction possibly.

edit: Ok, so, in d1.partitions it’s saved as 0xlabelid whereis sgdisk expects it as labelid

The original method used blkid -po udev which does not put 0x in front.

Shouldn’t be too hard to fix

[Sebastian Roth]

@Junkhacker Thanks for the picture. @Quazz Is exactly right about this commit having removed some of the UUID stuff as I noticed that we created an unnecessary file with UUID information that we have available in sfdisk output already. Therefore I went ahead and removed all the extra UUID stuff.

Can you please do me a favor and do a debug capture on your master client. When you get to the shell run blkid -po udev /dev/sda | grep "PART_TABLE_UUID" as well as sfdisk -d /dev/sda | grep "label-id" and post outputs here.

[Tom Elliott]

The error itself is really just a warning, but what’s unsettling is that it appears that the registry generation file and it fails due to space.

We may need to add some additional free space.

[Tom Elliott]

In particular the BUILDROOT configs managing these:

BR2_TARGET_ROOTFS_EXT2_SIZE="100M"
BR2_TARGET_ROOTFS_EXT2_INODES=0
BR2_TARGET_ROOTFS_EXT2_RESBLKS=5

Maybe we adjust to say 200M and give 100 reserved blocks? (Just thinking here.) Inodes should be plenty in regards to what is defaulted I think.

[george1421]

@Tom-Elliott said in The future of partclone and therefore FOG as it is:

Maybe we adjust to say 200M and give 100 reserved blocks?

I really don’t see a negative impact of this. It shouldn’t impact anything (other than give the filesystem a bit more room), since I believe this is the filesystem in RAM that gets allocated. I would say almost all systems to day have at least 1GB of ram, even tablets. You “might” run into issues with really old hardware that had 512MB of RAM or ARM based systems, but even then they usually start out with 1GB of RAM.

If you look at it from the initrd size it still shouldn’t make an impact on the init.xz file since this is “extra space” and that gets squished out when its compressed.

[Sebastian Roth]

@Tom-Elliott @george1421 Yes adding more room shouldn’t hurt. But could you please tell me where you see an error happening which leads you to think that there is a space issue within FOS? I can’t see it.

Edit: Never mind, I just saw the other topic. Building new inits with 256 MB size right now. Will take roughly four hours to complete. Will update the official binaries on the web server soon.
I wonder why I didn’t get (or notice) those errors when doing the tests on my VM?!

About the UUID issue, let’s first gather some more information before we decide what to do. @Junkhacker Would be great to get the two command outputs, thanks!

Please let us the UUID things here so we don’t loose the focus in this topic! @george1421 Did you figure out why you got different results than @Junkhacker when trying to deploy “old” images with partclone 0.3.12?

[george1421]

@Sebastian-Roth Sorry I didn’t have time yesterday because of some other issues and I’m traveling today. I’ll get back on the deployment differences on monday. Rebuilding my inits with 256MB size only added 15KB of size to the inits from my tests. So I don’t see a noticeable impact on size or delivery speeds. I don’t remember what the unpacked size differences were.