Include partclone 3.20?
-
@Fog_Newb Spoke too early. When I tried to build the FOS inits with partclone 0.3.20 I got some pretty ugly linking errors which I need more time to look into. Seems like some major change that needs to be addressed.
EDIT:
This commit in partclone added a dependency to openssl library libcrypto: https://github.com/Thomas-Tsai/partclone/commit/d4fb2259088268f1314b9246240bf45cbe7ed300Now it looks like libcrypto is not build fully static in our FOS inits (default buildroot) and that makes it fail when linking the partclone binaries.
Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)
Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG
-
@sebastian-roth said in Include partclone 3.20?:
Now it looks like libcrypto is not build fully static in our FOS inits (default buildroot) and that makes it fail when linking the partclone binaries.
I have been banging my head on this for two weeks. I’m not smart enough to fix it. But I did see it failing when trying to link to libcrypto. I was even thinking about taking .0.3.13 and just applying the apple file system code to it and see if it would compile. Not a nice solution but…
Clonezilla got around it by using the native ubuntu .deb files. As far as I can tell there is a conflict between the version of buidroot/libcrypto and partclone 0.3.20.
-
@george1421 I am sure we can get it compiled, either by switching buildroot to building all libs as static only (tried that but ran into other issues that would need to be fixed first) or we need to add some magic (patch, sed, buildroot-specific mk-foo) to make the linker happy.
Most probably going down the later path is way easier. I was able to manually link one of the binaries by adding
-ldlto the linker command. So it’s more or less a matter of adding this to partclone’s compilation per se. I will look into this some time this week.Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)
Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG
-
@sebastian-roth said in Include partclone 3.20?:
I was able to manually link one of the binaries by adding -ldl
This is the path I was going down too when researching the error they just said put -ldl as the last library in the linker. The question was where (ignorance of buildroot)? The other thing I thought is update the version of buildroot to current, maybe a later release of build root would have a compatible version of the crypto lib.
The other thing I found is that ubuntu has 0.3.20 with some kind of patch level beyond 0.3.20 but I couldn’t find the source files to see if building that had fixed the crypto lib errors.
-
@sebastian-roth Ok thanks, I will try it out and report back. These inits, they just go on the FOG server right? Or do I also need to add something to the mac OS Grub/FOS boot stick?
-
@fog_newb said in Include partclone 3.20?:
Or do I also need to add something to the mac OS Grub/FOS boot stick?
Which way did you create that boot stick?
Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)
Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG
-
This post is deleted! -
This post is deleted! -
@sebastian-roth Uhm… @george1421 can probably answer that better than I can.
Sorry for deleting and reposting the replies. The way the thread looks after replying, it looks like I replied to the OP and not to another reply.
-
@sebastian-roth said in Include partclone 3.20?:
Which way did you create that boot stick?
The OP used the tutorial I created to boot into grub and then into fos linux on a usb stick. But it sill uses the standard bzImage and init.xz. So no difference there. We still need buildroot to compile correctly.
-
Thanks, I wasn’t sure how this one was made. I downloaded the img and used rufus to write it onto a USB. It works great by the way.
-
@Fog_Newb There you go: https://github.com/FOGProject/fos/releases/tag/testing
Just download
init.xzand put in/var/www/html/fog/service/ipxe/in your FOG server - rename the original file instead of overwriting it! -
@fog_newb FWIW: the testing
init.xzgoes onto the flash drive if you are booting with that./var/www/html/fog/service/ipxe/is used when pxe booting via iPXE into the FOG menu. -
@sebastian-roth @george1421 Good news!
So far so good. I was able to capture the APFS drive on the real mac using the USB stick and the updated init.xy. I put the updated init.xy on both the stick and the server.
I was also able to capture both drives APFS and NTFS on the hackintosh laptop. Now I have to figure out how to back up the data in case deploying the images goes sideways.
-
@fog_newb Thanks for testing and the quick feedback.
Now I have to figure out how to back up the data in case deploying the images goes sideways.
Which data do you mean? On the device? Probably using Clonezilla to have a backup at hand?!?
-
@sebastian-roth Yes, Clonezilla but I need to get some more SSDs/HDDs.
-
@sebastian-roth Does 1.5.9.163 have these new inits?
I read the dev-branch info but wasn’t sure. I never know what most of that stuff means.
-
@fog_newb said in Include partclone 3.20?:
Does 1.5.9.163 have these new inits?
No these inits are still in testing phase. Since partclone is at the heart of FOG imaging and the versions jumped from 0.3.13 to 0.3.20 these inits must have more testing with people before they are added to the dev branch and then onto the 1.5.10 when its released. They might work great with your hardware but fail on some genericly assembled computer.
-
I’m currently testing init.xz images built by myself using the partclone-0320 branch. In addition, these images have, for my part, added these two commits of mine from github that fix bugs with BTRFS (https://github.com/FOGProject/fos/pull/47 https://github.com/FOGProject/fos/pull/45). The whole thing was built using Buildroot 2022.02.5, which fixes bugs related to udev (https://github.com/FOGProject/fos/issues/46). I know it’s too many changes to treat my experience as relevant to the addition of this particular partclone version, but I think it’s worth sharing anyway.
The FOG server on which this custom init.xz runs is based on Fedora 36 (/images on XFS), the latest (at the time of writing this) development version of FOG, compiled kernel 5.15.71, and an updated version of UDPcast to 20200328 (the latest does not work with FOG). This server, runs in production, so there are 30 images restored each day, and sometimes more (mostly Windows 10 and Ubuntu). But it happens to image other computers that have other systems, such as Fedora 36 Workstation with the BTRFS file system. I mainly use Multicast restore (computers in the school computer lab), but sometimes I need to restore one computer and I use Unicast.
The previous FOG server was based on Debian 10 (/images on EXT4), FOG 1.5.9 and the latest official kernel and init.xz. The situation has changed dramatically after the migration.
On the old system, using Unicast I was getting speeds of around 7-8GB/min (for both restore and capture). With Multicast, this speed increased to 12GB/min.
After I built the server on Fedora, I used init.xz from the partclone-0320 branch without patches and the official 5.15.68 kernel and the speed increased to 14GB/min using both Multicast and Unicast. The capture speed went up to 9GB/min. After changing init.xz to the current one, the speeds have not changed, but at least I can safely restore the BTRFS file system without any errors.
I don’t know how much of this is due to Partclone 3.20 and how much is due to migrating the system to Fedora with XFS, but I can say that so far Partclone 3.20 is running very stably and hasn’t crashed yet. And I have already restored images based on NTFS, XFS, EXT4 and BTRFS. If I only notice any flaws with the operation of the whole thing I will describe them. But so far I have no complaints about my system. If it continues to perform as well as it does now I will migrate the FOGs in other computer labs to what I am using in this particular one.
-
@piotr86pl Excellent feedback. While you have many accomplishments, it would be interesting in your buildroot configs to roll partclone back to 0.3.13 (only) and see if your side by side comparison still holds. This will kind of give the developers insight to if the partclone upgrade generated these improvements or it was updating buildroot to something current and optimized for linux 5.15.x,
Either well done rebuilding the FOG imaging environment and the performance gains you have.
