The future of partclone and therefore FOG as it is

[george1421]

@Junkhacker Well this is all good information. In your environment (except for the UUID issue) its deploying correctly. The other thing (thinking about all of the variables here) is that I’m running 1.5.4 on my production server. I’m pretty sure that 1.5.4 inits were created with a previous release of buildroot than what I’m building against. Its possible that buildroot updated applications too, but if everything appears to work in your environment this I can take one step to rule out buildroot differences causing this issue. As well it could be this old laptop that has issues. I’ll keep working on eliminating non-issues.

Thank you for your feedback.

[Sebastian Roth]

@george1421 Great stuff you are working on this full on!! I try to follow up on what you post and test. Though I don’t have the time to engage in this at the moment.

It’s interesting you and Junkhacker seem to get different results and it would be very important to figure out why. I can’t imagine it being a UEFI vs legacy difference issue as partclone should just be deploying the actual contents of the partitions no matter what the boot type or partition layout looks like. But what do I know…

@george1421 said:

I just tried to recapture a uefi golden image and I have an error about getPartitionLabel: command not found on line 486, then partclone exited with error code 139.

Not sure which version you used because we removed getPartitionLabel stuff recently. Please make sure you use the very latest version (master branch).

@Junkhacker The UUID error you see stems from an issue we had in the inits when I created the one you use for testing. This was fixed some weeks ago.

i tried the init you supplied, and i got a "failed to set disk guid (sgdisk -U) (restoreUUIDInformation)

Can you please post a picture of that?

[Junkhacker]

@george1421 more info for you, my dev server is 1.5.5 my prod is 1.4.4. not sure what versions the images files would have all been captured on, but i’ve tested 1.4.4 or earlier (not sure) though captures with 1.5.5.

[Junkhacker]

@Sebastian-Roth

[Tom Elliott]

That looks like an error code of some sort, weird.

[Junkhacker]

@Tom-Elliott yeah, definitely doesn’t look like a UUID (sorry the image is shrank and funny, was trying to get the image size down to upload)

[Quazz]

@Junkhacker The following commit changed UUID stuff https://github.com/FOGProject/fos/commit/f49b0f7d5b0c90866a6dcdbbd5d59e529857242e

On MBR, the label-id is in the form of what appears to be an error code (0xrandomnumbersandletters) hence the output that you saw.

I don’t know for sure why it would fail, but something in that direction possibly.

edit: Ok, so, in d1.partitions it’s saved as 0xlabelid whereis sgdisk expects it as labelid

The original method used blkid -po udev which does not put 0x in front.

Shouldn’t be too hard to fix

[Sebastian Roth]

@Junkhacker Thanks for the picture. @Quazz Is exactly right about this commit having removed some of the UUID stuff as I noticed that we created an unnecessary file with UUID information that we have available in sfdisk output already. Therefore I went ahead and removed all the extra UUID stuff.

Can you please do me a favor and do a debug capture on your master client. When you get to the shell run blkid -po udev /dev/sda | grep "PART_TABLE_UUID" as well as sfdisk -d /dev/sda | grep "label-id" and post outputs here.

[Tom Elliott]

The error itself is really just a warning, but what’s unsettling is that it appears that the registry generation file and it fails due to space.

We may need to add some additional free space.

[Tom Elliott]

In particular the BUILDROOT configs managing these:

BR2_TARGET_ROOTFS_EXT2_SIZE="100M"
BR2_TARGET_ROOTFS_EXT2_INODES=0
BR2_TARGET_ROOTFS_EXT2_RESBLKS=5

Maybe we adjust to say 200M and give 100 reserved blocks? (Just thinking here.) Inodes should be plenty in regards to what is defaulted I think.

[george1421]

@Tom-Elliott said in The future of partclone and therefore FOG as it is:

Maybe we adjust to say 200M and give 100 reserved blocks?

I really don’t see a negative impact of this. It shouldn’t impact anything (other than give the filesystem a bit more room), since I believe this is the filesystem in RAM that gets allocated. I would say almost all systems to day have at least 1GB of ram, even tablets. You “might” run into issues with really old hardware that had 512MB of RAM or ARM based systems, but even then they usually start out with 1GB of RAM.

If you look at it from the initrd size it still shouldn’t make an impact on the init.xz file since this is “extra space” and that gets squished out when its compressed.

[Sebastian Roth]

@Tom-Elliott @george1421 Yes adding more room shouldn’t hurt. But could you please tell me where you see an error happening which leads you to think that there is a space issue within FOS? I can’t see it.

Edit: Never mind, I just saw the other topic. Building new inits with 256 MB size right now. Will take roughly four hours to complete. Will update the official binaries on the web server soon.
I wonder why I didn’t get (or notice) those errors when doing the tests on my VM?!

About the UUID issue, let’s first gather some more information before we decide what to do. @Junkhacker Would be great to get the two command outputs, thanks!

Please let us the UUID things here so we don’t loose the focus in this topic! @george1421 Did you figure out why you got different results than @Junkhacker when trying to deploy “old” images with partclone 0.3.12?

[george1421]

@Sebastian-Roth Sorry I didn’t have time yesterday because of some other issues and I’m traveling today. I’ll get back on the deployment differences on monday. Rebuilding my inits with 256MB size only added 15KB of size to the inits from my tests. So I don’t see a noticeable impact on size or delivery speeds. I don’t remember what the unpacked size differences were.

[Sebastian Roth]

@george1421 Any news on this topic? I was hoping to engage more in this but as you see things keep popping up that need quick fixing…

[george1421]

@Sebastian-Roth Not with a resolution as of now. But here is an update with good to know stuff.

I did a git pull on the fos github and updated my local repository with the master. Then without thinking overwrote all of the edits I did to the scripts in my rootfs_overlay directory. Not what I wanted to do but I rebuilt the inits without any of my edits and only with the 0.3.12 partclone build. Unfortunatly it still does the same with Win7 unable to boot.

BUT along the way I discovered something.

Per our earlier discussion I change the buildroot setting of

BR2_TARGET_ROOTFS_EXT2_SIZE="100M"

to

BR2_TARGET_ROOTFS_EXT2_SIZE="256M"

When I pxe booted into FOS I received the following error:

RAMDISK: incomplete write (-28 != 4096)
XZ-compressed data is corrupt
Kernel panic - not syncing: VS: unable to mount root fs on unknown

I went into FOG Configuration-FOG Settings->TFTP Server->KERNEL RAMDISK SIZE and viewed the size. The size default is 127000. I had the setting at 255000 because I was testing something earlier. This 255MB is of course smaller than the ROOTFS size I set in build root of 256MB. So the decompression failed and the kernel panicked because it couldn’t mount the virtual hard drive. Setting KERNEL RAMDISK SIZE to 300000 resolved the issue. I did not test any smaller to see where it started to fail due to time restrictions.

[Junkhacker]

@george1421 said in The future of partclone and therefore FOG as it is:

Not what I wanted to do but I rebuilt the inits without any of my edits and only with the 0.3.12 partclone build.

you must edit the funcs.sh file to remove the --ignore_crc flags or 0.3.12 will not work properly

[Sebastian Roth]

@george1421 said in The future of partclone and therefore FOG as it is:

RAMDISK: incomplete write (-28 != 4096)
XZ-compressed data is corrupt

Yes, ran into that yesterday already. See here. We moved back to 101 MB (just a little larger so that hopefully we don’t run into a space issue again) for 1.5.6 binaries. But we’ll move to 256 MB for 1.5.7 in the near future I suppose.

[george1421]

@Sebastian-Roth Additional testing with 257MB == fail. I figured 100MB and fog’s default was 127MB, so I went 256MB with a 275MB ram drive and success.

@Junkhacker Thank you for catching that. I fixed the func.sh script and reran deployment with still a failure (to the same laptop). So after lunch I’ll take a different computer and deploy to it just to rule out this laptop hardware as a fault domain.

[Junkhacker]

@george1421 when it fails to work for you, do you see any unexpected behavior in the imaging process?

[george1421]

@Junkhacker In the imaging process it self no, when the drivers go to copy over it says no room left on device or something like that. But that is only a file copy. I’m suspecting the disk format is damaged by then because of the partclone thing. I’m far from done testing on this. If new hardware give the same results then I’ll go back and create the inits using 0.2.89 (or whatever was original) to ensure I don’t have something else breaking the imaging deployment.