How to manually upload an existing image
Say I have a few images made using CloneZilla. The folder structure looks like this:
user@myserver:~/partimag$ tree -ash . ├── [4.0K] image-name │ ├── [ 677] blkdev.list │ ├── [ 249] blkid.list │ ├── [6.3K] clonezilla-img │ ├── [ 159] dev-fs.list │ ├── [ 4] disk │ ├── [ 20K] Info-dmi.txt │ ├── [ 15K] Info-lshw.txt │ ├── [2.0K] Info-lspci.txt │ ├── [ 216] Info-packages.txt │ ├── [ 90] Info-saved-by-cmd.txt │ ├── [ 10] parts │ ├── [ 33] sda1.info │ ├── [8.8M] sda1.ntfs-ptcl-img.gz.aa │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.aa │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ab │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ac │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ad │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ae │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.af │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ag │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ah │ ├── [1.5G] sda2.ntfs-ptcl-img.gz.ai │ ├── [ 37] sda-chs.sf │ ├── [1024K] sda-hidden-data-after-mbr │ ├── [ 512] sda-mbr │ ├── [ 333] sda-pt.parted │ ├── [ 295] sda-pt.parted.compact │ └── [ 190] sda-pt.sf etc...
FOG Images, however, look like this:
user@myserver:/images$ tree . ├── an-old-image │ ├── d1.fixed_size_partitions │ ├── d1.original.fstypes │ ├── d1.original.partitions │ ├── d1.original.swapuuids │ ├── rec.img.000 │ └── sys.img.000 ├── a-newer-image │ ├── d1.fixed_size_partitions │ ├── d1.mbr │ ├── d1.minimum.partitions │ ├── d1.original.fstypes │ ├── d1.original.swapuuids │ ├── d1p1.img │ ├── d1p2.img │ └── d1.partitions etc...
Could I somehow import the CloneZilla images into FOG, without deploying and recapturing the image?
If the CloneZilla image is not compressed, it will not have the
.gzextention (easy to check for), and the last command becomes:
cat /home/user/czimg/sda2.ntfs-ptcl-img.* | pigz -stdout > /images/fogimg/d1p2.img
If CloneZilla was invoked using dd, we can concatenate and pipe to partclone:
cat /home/user/czimg/sda2.ntfs-dd-img.* | partclone.$fstype -fsck-src-part -c -s - -O - | pigz -c > /images/fogimg/d1p2.img
etc… There might be a few more cases to consider.
@Tom-Elliott Or if something besides beginner mode is used in CloneZilla
@dolf Maybe, with this addition, I can create some code that can do this automagically? May not be a while but I think it could be very useful. Of course I don’t know if gunzip will work if say the CloneZilla image is not already compressed.
Feel free to put this in the wiki if you think it’s worthy.
I’m working with a Windows 7 installation, where sda1 is a small boot partition and sda2 is the large partition called
- Use GParted (from your favourite linux disc or GParted Live) to resize sda2 to a minimum.
- Use CloneZilla to capture the disk. I used beginner mode with the savedisk option. I transferred it to
/home/user/czimg/…on the FOG server over ssh, but you could use a USB HDD or any other method.
- Create a new resizable image on the FOG web interface. Note the image location.
- Create the location specified in the previous step, e.g.
- Do magic:
cp /home/user/czimg/sda-pt.sf /images/fogimg/d1.minimum.partitions cp /home/user/czimg/sda-pt.sf /images/fogimg/d1.partitions cp /home/user/czimg/sda-mbr /images/fogimg/d1.mbr echo "1" > /images/fogimg/d1.fixed_size_partitions echo "/dev/sda2 ntfs" > /images/fogimg/d1.original.fstypes cp /home/user/czimg/sda1.ntfs-ptcl-img.gz.aa /images/fogimg/d1p1.img cat /home/user/czimg/sda2.ntfs-ptcl-img.gz.* > /images/fogimg/d1p2.img
The last command will take the longest. If you want to see something happening, install pipe viewer and replace the last command with:
cat /home/user/czimg/sda2.ntfs-ptcl-img.gz.* | pv > /images/fogimg/d1p2.img
@dolf Could also be a CAT 5 cable somewhere.
@dolf Wow awesome man, way to go! I had serious doubts about this being easy!
Can you please lay out details on all your steps to help people in the future?
It works! The minimum partition size is probably wrong, but I will only deploy to larger drives in any case.
I created a new image in the Fog web interface, copied the d1.* files from a very similar image, and replaced d1p*.img with concatenated versions of the CloneZilla images. Currently deploying, and
partcloneseems happy thus far. Just wondering whether the resize will work…
By the way:
Sorry for the delayed responses. It typically takes me 45 minutes to deploy a 65GB image. Without compression the shortest time should be around
((65×1024)MB ÷ (100÷8)MB/s) ÷ 60s = 88 minutes. There’s a 100MB switch somewhere between here and the server
OK so while the imaging runs, I’m reading code.
CloneZilla saves the partition like this (from the log file
partclone.ntfs -z 10485760 -N -L /var/log/partclone.log -c -s /dev/sda2 --output - | pigz -c --fast -b 1024 -p 16 --rsyncable | split -a 2 -b 4096MB - /home/partimag/2016-07-12-21-img-eelabtest/sda2.ntfs-ptcl-img.gz. 2> /tmp/split_error.TMVF3J
Fog does it like this (from
partclone.$fstype -fsck-src-part -c -s $part -O $fifoname -N -f 1
I’m not really sure how the FIFO buffer works, but it’s called
/tmp/pigz1, so I guess it’s the same as piping the output of
pigz, which is exactly what CloneZilla does. Therefore, maybe this would be sufficient to “convert from CloneZilla format to Fog format”:
cat sda2.ntfs-ptcl-img.gz.* > d1p2.img
In other words, no recompression is needed. Am I missing something here?
@dolf I’d urge you to try a non-resizable image in fog first. See if it works. If all the computers are identical they will all have the same size HDD. It’ll either work or not. If it works, that’s how you do it for this model.
dolf last edited by dolf
WOW thanks for the advice. I’ll try that soon.
I think the 1MB space between the partitions is just for alignment.
@dolf Ok, I had a quick look through those - seems very simple, I like it.
disk- appears to be a list of all disks. this one just says
sda-pt.partedappear to be similar to an
sda-mbris clearly a copy of the mbr.
sda-chs.sfis information about the disk’s cylinders, heads, and sectors.
sda1.infois just saying that’s the boot partition.
partsis a list of partitions. This one has just
info-saved-by-cmd.txtappears to be output from a command CZ uses.
info-packages.txtis a list of exact packages that build of CZ used.
the info files are info about the computer it was pulled from.
dev-fs.listis a list of partitions and their format.
clonezilla-imgis literally the output that CZ gave during the image it looks.
blkid.listis simply the output of
blkdev.listI assume is the output of a package called
Really, to convert this I think all you need is the information from the
sda-pt.sffile. This would be a non-resizable image in fog. Use an existing non-resizable image as a template to help guide you with putting the numbers in the right spots. Change
d1.mbrand then you need to uncompress and mount the image, and then capture+compress with partclone and place all files where they should be, assign permissions, and probably fail a few times before it’s right.
Of course the extra info between the mbr and partitions is what’s really interesting. It could just be meaningless gibberish in unused space, but maybe not…
CloneZilla files, excluding the large ones.
Oh boy… While I’m sure it’s possible, someone is going to have to become familiar with how CloneZilla stores images to do it. To be honest - it’d probably be easier to stick with FOG and get it working and spread what you learn so the Devs can consider it and so others can read it. But I’ll try to point you in the right direction.
Just glancing at the files, the 3.8GB files are obviously gun-zipped chunks. Those will need decompressed into one big image file. You’ll need to mount that via NTFS-3G in a Linux system, and then manually run the Partclone command that would take the image and compress it. This of course is just for partitions 1 and 2.
sda-mbris obviously the MBR, and its 512 bytes. That can just be renamed to
d1.mbrand then placed. The files parts
partssound important, you should post the contents of those.
And, 1,024 bytes worth of info after the MBR but before partition 1 sounds interesting, the
sda-hidden-data-after-mbrfile. That could honestly be the missing piece for your other thread: https://forums.fogproject.org/topic/8059/pc-unbootable-after-capture-fails/14
Additional reading on the subject of manually manipulating images for use with FOG: https://forums.fogproject.org/topic/7432/mount-and-extract-files-from-images?page=1