The future of partclone and therefore FOG as it is

[Quazz]

@george1421 Did you remove the partclone patch file?

[george1421]

@Quazz Yes, also I attached the changes I made to the Config.in and partclone.mk early in this thread.

So what I’m after at the moment is to build a truth table. I’m saying that 0.3.12 doesn’t deploy the older image files correctly, Junkhacker is saying with my inits it does work. I need a few more people to test to confirm one way or the other. If my inits work on everyone else’s environment than mine, I need to dig into why. Its not to say one is right or one is wrong, its to understand why. Because if I’m the odd one with issues, there will be at least a handful of others with the same issue. So its good we have problems like this so we can understand before the update hits everyone’s FOG server.

[Quazz]

@george1421 Of course, just trying to figure out if there’s any differences in the inits themselves that might cause a disrepancy.

I’ll let you know if it deploys correctly on my test system tomorrow.

Do you have a funky partition layout perhaps? I’m assuming fairly vanilla, but you never know.

[Junkhacker]

@george1421 i just tested a deploy with the init i downloaded from your google drive link today. it deployed and booted (though i did get the UUID error, so you need to update your files to the latest build, not that it’s related to the partclone issues)

I’m at kind of a loss as to what’s wrong at this point

[Quazz]

@george1421 Deployed successfully with your init on my end, W10 Pro UEFI.

[george1421]

@Quazz Thank you for checking, I guess I need to better understand what is going wrong with my deployments. They work perfectly with the 0.2.89 partclone even when I build the inits.

[Junkhacker]

@george1421 for reference: I tested with your inits using both bios and uefi configurations using a physical computer and a vmware VM. I had no problems with booting the systems after imaging. all images were compressed with zstd (not that that should make a difference, but putting it out there) and the images ranged from windows 7 to windows 10. imaging done while in uefi mode seemed significantly slower. i don’t know if that’s normal, since i don’t normally use that.

[Junkhacker]

@george1421 have you had a chance to do any further testing? did you build 32bit versions and have you tried using them?

[george1421]

@Junkhacker Sorry I haven’t had time between traveling and project deadlines most everything else has got put on back burner. Both you and Quazz say they are working fine. I’m not sure why my images are not. I know at least one I tested with was created with 1.5.4 the rest may be 1.4.4, but I’d have to look at the image creation date to be sure. Everything looks like it deploys correctly with partclone, just when I try to mount the partition and copy the drivers over it says no space left on the device.

[Junkhacker]

@george1421 No problem. I was just hoping to make progress on this. I found this https://github.com/stevenshiau/clonezilla/issues/31 and wondered if it was related since it was from when clonezilla updated their partclone version.

[Quazz]

@george1421 Well, potentially that was a resize bug in the inits if you’re using an NVME drive.

Could also be the mount failing for some reason?

Just some ideas.

[Quazz]

@george1421 Well, we did find a resize bug in the inits, particularily on fast disks.

Might be time to revive this idea soon and test it out again.

[george1421]

@Quazz Great idea.

I’m rebuilding the inits with partclone 0.3.13 and zstd 1.4.0 using the FOG 1.5.7 base files for the inits. I suspect trouble the first time through compiling this since I’ve forgotten everything that I knew from before. Plus I need to update my test fog server to 1.5.7 so it will be a challenging morning to say the least.

@Sebastian-Roth Are you still using buildroot v2019.02.1 for the inits?

[Quazz]

@george1421 I don’t believe FOS github has the fix included yet at this point, though shouldn’t be too long.

Still trying to figure out if/when/where the problem exactly started.

As far as I could find it’s supposedly been around for ages (sfdisk - udev conflict), but isn’t necessarily triggered usually.

[Sebastian Roth]

@george1421 Yeah Quazz is right. I was in a bit of a rush to get some minor issues fixed when releasing the inits and did not push the changes to github yet. I will sort all this in a couple of hours though. Don’t rush into testing partclone yet as we might even think about moving to the latest buildroot in the next days too.

[george1421]

@Sebastian-Roth OK I’ll hold off until I get the green light from you.

[Quazz]

I’m preparing a build with Buildroot version 2019.02.4 and one with that Buildroot version + partclone 3.12, will test and post them on Monday probably.

[george1421]

@Quazz Here are the edits (not in diff format but george shorthand) that junkhacker recommended to take advantage of new features in pigz and zstd for data dedup systems

vi fog.upload
partclone.imager -aX0 -c -s "$hd" -O /tmp/pigz1 -N -f 1

vi funcs.sh
pigz --rsyncable $PIGZ_COMP < $fifo | split -a 3 -d -b 200m - ${file}. &
pigz --rsyncable $PIGZ_COMP < $fifo > ${file}.000 &

zstdmt --rsyncable -B128 --ultra $PIGZ_COMP < $fifo | split -a 3 -d -b 200m - ${file}. &
zstdmt --rsyncable -B128 --ultra $PIGZ_COMP < $fifo > ${file}.000 &

partclone.$fstype -aX0 -n "Storage Location $storage        , Image name $img" -cs $part -O $fifoname -Nf 1

remove --ignore_crc

[Quazz]

@george1421 Thanks, I had already done some of that, got all of it in there now.

[Sebastian Roth]

@george1421 said in The future of partclone and therefore FOG as it is:

OK I’ll hold off until I get the green light from you.

Totally forgot to let you know that it’s been done - weeks ago already (ref).