Custom init.gz
-
Alright, as I’m working on GUI lately, I’ve left buildroot alone for a little bit. I was finally able to build the partclone binaries on my system, but I can’t figure out how to get them built native to the buildroot process. For now, though I’ll just recreate the fogpartinfo within the fog.mk.
Though I’m sure we’ll have a need for these, eventually I’ll sit to figure this out. Remember I’m building on CentOS 6.4, and finding all the dependencies is not always that easy, I’m sorry. -
I think it’s starting to make sense, why, the init.gz file will not perform the upload task on Single Disk, Resizable. When the image is created, it does not take a snapshot of the MBR in use. This, by itself, is not the issue, but when you go to deploy the image, one of the first steps needed is to remove the partitioning schemes. This also removes the MBR, then it tries to create the new partitions, but the MBR that gets used doesn’t exist, so the system can’t see the second partition at all.
Just me two cents to help me in fixing this issue.
-
Hi
There are indeed some needed work both on the kernel and the init.gz script. However, we have to care not to break anything… As of now, I think we should have an option for 64 bits install… And perhaps add more libs & tools in it, though the latest version from SVN has already some really helpful tool. One thing that was removed, and broke compatibility was the probable removal of the “loadkeys” utility, that renders the FOG_KERNEL_KEYMAP useless.
I’m also a little bit confused about how the init.gz is generated from SVN… The doc to build a custom one is around here: [url]http://www.fogproject.org/wiki/index.php/Modifying_the_Init_Image[/url]
but I’m not sure I’ve found how to properly generate it in SVN from src/buildroot or what not… -
Gilou,
The src/buildroot files you’re seeing are only provided as an opensource tool to build your own initrd.gz (init.gz) file. When a user performs an install, the init.gz file is copied from the <folder_where_stored>/src/tftp directory which if you look in the tftp, you see the sub folders fog, kernel, images.
-
I know that, but my point is actually, how is it supposed to be used ? I didn’t look too far, but if we want to improve this, the toolchain should be documented
-
I’m attempting something dangerous I think.
I’m attempting to build a 64 bit init.gz, which of course will require a 64 bit kernel to go along with it. I’ll try to keep it posted here.
-
Well,
Good news/bad news time.
As I don’t know fully what I’m doing in creating this, I got the init.gz to actually run and operate with 64 bit capabilities. The kernel and init.gz were fully compatible. I didn’t notice anything major, except… all the libraries could not be found
-
Good news, sorta…
I got the libraries needed and the partclone is working. It isn’t giving the ncurses menu (the menu that shows the progress) but it is creating the (at least the sys file) right now. I haven’t tested the abilities for single disk resizeable, but I do have ntfs working on a multipart single disk image so far. I haven’t tested deploy, but at least upload seems to be working.
With this, the init.gz is very large comparitively to my previous version. By large, I mean it spiked from 6.1 mb from my original source to 30mb with all libraries attached. However, it has name resolution and even includes scp and ssh utilities.
-
Now I’m attempting to rebuild my tarball and have integrated the scripts to install the libraries needed automatically. This way I can just build it. It contains the original .config file for 32 bit builds and will require make ARCH=i386 to operate. If you try to build it with x86_64, it will most likely fail on the uclibc steps as there’s no directory for lib64 libraries in uclibc. I’m still testing so don’t attempt it yet, but the fix seems to be switching from uclibc to eglibc (experimental) for this. I’m still testing though but thinking this is the best method so far.
-
I’m running into some weird issues that is, I think, requiring me to perform a debian based build process. Partclone is almost entirely only supported (building the source this is) on debian. I think part of my issues so far is because it’s requiring shared libraries (being built without --enable-static) however if I try to run build with --enable-static, it fails miserably on CentOS 6.4 64 bit and i386 bit.
I’m just trying to get a hand at what’s happening.
-
Hi Tom keeping a close eye on ur work on the init.gz I really like where ur going with it. give us a shout when it ready for testing.
[SIZE=6][B]Thanks for all your hard work ;)[/B][/SIZE] -
There’s quite a few modifications from the 2011.08 package that FOG 0.33b is built on.
First things first, if you’re going to try updating it, make sure you have a debian system (vm or not) to build partclone on.
To build partclone I use:
Download the partclone you want. I’m using partclone_0.2.58.orig.tar.gz
Extract the file:
[code]tar -xzf partclone_0.2.58.orig.tar.gz[/code]Prep the system.
[code]echo “deb http://free.nchc.org.tw/drbl-core drbl dev stable testing unstable” >> /etc/apt/sources.list
echo “deb-src http://free.nchc.org.tw/drbl-core drbl dev stable testing unstable” >> /etc/apt/sources.list
apt-get update
apt-get build-dep partclone
cd partclone-0.2.58/[/code]Begin the Makefile creation: If you arch is already i386/i686 this will work.
[code]./configure --enable-static --enable-xfs --enable-btrfs --enable-ntfs --enable-extfs --enable-fat --enable-hfsp --enable-static --enable-ncursesw[/code]I don’t know what kinds of problems you’ll run into, so I’m going to just assume for now that you’re on i386 debian.
Build the binaries:
[code]make[/code]Once complete, change to the src directory of partclone.
[code]cd src[/code]Now copy the partclone files to your buildroot fog folder: (I think it something like below)
[code]for i inls|grep ^partclone\\\.|grep -v \\\.c$|grep -v \\\.h$|grep -v \\\.o$; do cp $i ~/buildroot-2013.08.1/package/fog/partclone/; done[/code]Your partclone binaries should be ready for use on the init.gz system now.
-
Eventually I’ll update the trunk/src/buildroot to contain the needed elements for the buildroot portions of the package, to include the Config.in file to make things work properly.
For now, it’s just a hoopla of a lot of work.
-
Thanks tom going to give this a go :rolleyes:
-
So,
In trying to get partclone working, I’ve noticed a couple of things.
First,
I forgot all about my old sector based partition creation to actually recreate /dev/sda2 during upload process. The only problem I forsee with using sector based partition creation is 4k disks. I can probably figure out a method to get the sector autonomously in the future so this wouldn’t necessarily be a problem anymore, but for now I’m back to using that.
After the partition is resized, it needs (I think) to be fixed.
Something like:
[code]ntfsfix -b -d ${part}[/code]Of course you need the partition recreated and existing for this to work, but that should be trivial.
-
Figured out a couple of things.
The display of partclone wasn’t working because it was counting all output types as errors even when it wasn’t an error (in fog script it redirects errors to status.fog.)
Removing the redirects seems to allow display to work.
The next problem to overcome, after upload is complete, is download of resizeable images. I think this issue, as described earlier, was due to the sector/kB associations. First, ntfsprogs DOES NOT like 100% as a layout. -1s seems to work for the end point, but I prefer to use actual size of the disk. The method to this approach, assuming Windows 7 always uses the same information for the 1st partition, is to, rather than using kB as the startpoint, is to use B for the start point. Sectors will work as well, 206847 is the end sector of the first partition. However, the current method used kB. So the start kB of the second partition was/is always 105906kB, which for all purposes is the same end kB of the first partition. I was able to create the second partition properly by using the diskSize variable, slightly modified (added grep -v “Flags”) as the end point with kB and the start point as 105906176B.
Hope this helps out. Partclone seems to be working on the upload side, now just waiting to play with download. Hopefully I’ll be able to post something soon.
-
Well,
I’ve now tested upload of ext partclone, my Debian box, and that seems to have taken well. Now to deploy the same methodology for ntfs test. I’ve actually got a real system to test this on, so I’m going to test on both. The real system will require generalized sysprep as it’s, obviously, different than my VM, but all and all, I will keep all posted.
-
It seems the upload process worked, but the deploy did not work properly. I think this is because the win7.mbr in use has something wrong with it and can’t get to the proper winload, or I’m a total idiot. However, I’m going to do the same test once again, but with a sysprepped system rather than just a generic test. As an aside to this all, it appears the multipart images nonsysprepped is working once again now that I’m using partclone. Maybe I’m overthinking it, but essentially we’re back to using a cloning utility that works in much the same way we’re all used to.
-
All,
I think I’ve tweaked enough with the init.gz. Still in initial testing, but I think I’ve got it down enough that it was safe for me to update the src/buildroot files on the trunk of svn. We’re now at r1004 because of this. Now once I’m complete with testing the init.gz I’ll update svn with my latest and greatest one. It’s built around 3.12.1 headers so I may have to make a go at updating the kernel that we can download.
Compression,
I’m going to move mine away from gzip as it’s, to me , a rather large init.gz file if only gzipped. I’m compressing it with
[code]lzma -z -9 -e < rootfs.ext2 > init.gz[/code]
Thank you Fernando for giving me the insight for this compression medium.as it makes it from ~10-12 MB down to ~5-6 MB. This means less time waiting for it to load.
I don’t mind the kernel being larger as that is to be expected due to it containing all the drivers for the systems to be imaged, but I think a smaller loading rootfs is a necessary thing.
-
I’ve added a few tools to this. sed, rsync, librsync, resize2fs, gawk, and a few others that I can’t remember off the top of my head.
The lzma compression is still used, and the init.gz size is now around 7.5 MB. I hope that’s okay with everyone.
This has been really fun, if not frustrating at times, to play with. I haven’t figured out a way to send the output of Partclone to the screen and only send errors to /tmp/status.fog. I also haven’t, quite yet, figured out how to enable the progress bar on the Web GUI, but that can be for later, or never if it comes down to it. Having the screen tell is is good enough for me, though it wouldn’t hurt to know the progress information from time to time.
I have saved quite a lot of loading time on the kernel though. I’ve figured out a method (though the configuration file isn’t updated with this new method quite yet) to get the kernel size down to just shy of 6MB. This means our total load out size is only around 14 MB versus the old 0.33 sizes of around 12-13MB for the kernel and another 12 MB for the init.gz. 14 vs. 24, you be the judge.
