How to manually upload an existing image
-
Say I have a few images made using CloneZilla. The folder structure looks like this:
user@myserver:~/partimag$ tree -ash . ├── [4.0K] image-name │ ├── [ 677] blkdev.list │ ├── [ 249] blkid.list │ ├── [6.3K] clonezilla-img │ ├── [ 159] dev-fs.list │ ├── [ 4] disk │ ├── [ 20K] Info-dmi.txt │ ├── [ 15K] Info-lshw.txt │ ├── [2.0K] Info-lspci.txt │ ├── [ 216] Info-packages.txt │ ├── [ 90] Info-saved-by-cmd.txt │ ├── [ 10] parts │ ├── [ 33] sda1.info │ ├── [8.8M] sda1.ntfs-ptcl-img.gz.aa │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.aa │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ab │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ac │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ad │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ae │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.af │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ag │ ├── [3.8G] sda2.ntfs-ptcl-img.gz.ah │ ├── [1.5G] sda2.ntfs-ptcl-img.gz.ai │ ├── [ 37] sda-chs.sf │ ├── [1024K] sda-hidden-data-after-mbr │ ├── [ 512] sda-mbr │ ├── [ 333] sda-pt.parted │ ├── [ 295] sda-pt.parted.compact │ └── [ 190] sda-pt.sf etc...
FOG Images, however, look like this:
user@myserver:/images$ tree . ├── an-old-image │ ├── d1.fixed_size_partitions │ ├── d1.original.fstypes │ ├── d1.original.partitions │ ├── d1.original.swapuuids │ ├── rec.img.000 │ └── sys.img.000 ├── a-newer-image │ ├── d1.fixed_size_partitions │ ├── d1.mbr │ ├── d1.minimum.partitions │ ├── d1.original.fstypes │ ├── d1.original.swapuuids │ ├── d1p1.img │ ├── d1p2.img │ └── d1.partitions etc...
Could I somehow import the CloneZilla images into FOG, without deploying and recapturing the image?
-
Oh boy… While I’m sure it’s possible, someone is going to have to become familiar with how CloneZilla stores images to do it. To be honest - it’d probably be easier to stick with FOG and get it working and spread what you learn so the Devs can consider it and so others can read it. But I’ll try to point you in the right direction.
Just glancing at the files, the 3.8GB files are obviously gun-zipped chunks. Those will need decompressed into one big image file. You’ll need to mount that via NTFS-3G in a Linux system, and then manually run the Partclone command that would take the image and compress it. This of course is just for partitions 1 and 2.
sda-mbr
is obviously the MBR, and its 512 bytes. That can just be renamed tod1.mbr
and then placed. The files partssda1.info
andparts
sound important, you should post the contents of those.And, 1,024 bytes worth of info after the MBR but before partition 1 sounds interesting, the
sda-hidden-data-after-mbr
file. That could honestly be the missing piece for your other thread: https://forums.fogproject.org/topic/8059/pc-unbootable-after-capture-fails/14Additional reading on the subject of manually manipulating images for use with FOG: https://forums.fogproject.org/topic/7432/mount-and-extract-files-from-images?page=1
-
CloneZilla files, excluding the large ones.
-
@dolf Ok, I had a quick look through those - seems very simple, I like it.
disk
- appears to be a list of all disks. this one just sayssda
sda-pt.sf
,sda-pt.parted.compact
, andsda-pt.parted
appear to be similar to anfdisk -l
output.
sda-mbr
is clearly a copy of the mbr.
sda-chs.sf
is information about the disk’s cylinders, heads, and sectors.
sda1.info
is just saying that’s the boot partition.
parts
is a list of partitions. This one has justsda1
andsda2
.
info-saved-by-cmd.txt
appears to be output from a command CZ uses.
info-packages.txt
is a list of exact packages that build of CZ used.
the info files are info about the computer it was pulled from.
dev-fs.list
is a list of partitions and their format.
clonezilla-img
is literally the output that CZ gave during the image it looks.
blkid.list
is simply the output ofblkid
blkdev.list
I assume is the output of a package calledblkdev
Really, to convert this I think all you need is the information from the
sda-pt.sf
file. This would be a non-resizable image in fog. Use an existing non-resizable image as a template to help guide you with putting the numbers in the right spots. Changesda-mbr
tod1.mbr
and then you need to uncompress and mount the image, and then capture+compress with partclone and place all files where they should be, assign permissions, and probably fail a few times before it’s right.Of course the extra info between the mbr and partitions is what’s really interesting. It could just be meaningless gibberish in unused space, but maybe not…
-
WOW thanks for the advice. I’ll try that soon.
I think the 1MB space between the partitions is just for alignment.
-
@dolf I’d urge you to try a non-resizable image in fog first. See if it works. If all the computers are identical they will all have the same size HDD. It’ll either work or not. If it works, that’s how you do it for this model.
-
OK so while the imaging runs, I’m reading code.
CloneZilla saves the partition like this (from the log file
clonezilla-img
partclone.ntfs -z 10485760 -N -L /var/log/partclone.log -c -s /dev/sda2 --output - | pigz -c --fast -b 1024 -p 16 --rsyncable | split -a 2 -b 4096MB - /home/partimag/2016-07-12-21-img-eelabtest/sda2.ntfs-ptcl-img.gz. 2> /tmp/split_error.TMVF3J
Fog does it like this (from
savePartition()
insrc/buildroot/package/fog/scripts/usr/share/fog/lib/funcs.sh
)partclone.$fstype -fsck-src-part -c -s $part -O $fifoname -N -f 1
I’m not really sure how the FIFO buffer works, but it’s called
/tmp/pigz1
, so I guess it’s the same as piping the output ofpartclone
throughpigz
, which is exactly what CloneZilla does. Therefore, maybe this would be sufficient to “convert from CloneZilla format to Fog format”:cat sda2.ntfs-ptcl-img.gz.* > d1p2.img
In other words, no recompression is needed. Am I missing something here?
-
I created a new image in the Fog web interface, copied the d1.* files from a very similar image, and replaced d1p*.img with concatenated versions of the CloneZilla images. Currently deploying, and
partclone
seems happy thus far. Just wondering whether the resize will work…By the way:
Sorry for the delayed responses. It typically takes me 45 minutes to deploy a 65GB image. Without compression the shortest time should be around((65×1024)MB ÷ (100÷8)MB/s) ÷ 60s = 88 minutes
. There’s a 100MB switch somewhere between here and the server -
It works! The minimum partition size is probably wrong, but I will only deploy to larger drives in any case.
-
@dolf Wow awesome man, way to go! I had serious doubts about this being easy!
Can you please lay out details on all your steps to help people in the future?
-
@dolf Could also be a CAT 5 cable somewhere.
-
Feel free to put this in the wiki if you think it’s worthy.
I’m working with a Windows 7 installation, where sda1 is a small boot partition and sda2 is the large partition called
C:
.- Use GParted (from your favourite linux disc or GParted Live) to resize sda2 to a minimum.
- Use CloneZilla to capture the disk. I used beginner mode with the savedisk option. I transferred it to
/home/user/czimg/…
on the FOG server over ssh, but you could use a USB HDD or any other method. - Create a new resizable image on the FOG web interface. Note the image location.
- Create the location specified in the previous step, e.g.
mkdir /images/fogimg
- Do magic:
cp /home/user/czimg/sda-pt.sf /images/fogimg/d1.minimum.partitions cp /home/user/czimg/sda-pt.sf /images/fogimg/d1.partitions cp /home/user/czimg/sda-mbr /images/fogimg/d1.mbr echo "1" > /images/fogimg/d1.fixed_size_partitions echo "/dev/sda2 ntfs" > /images/fogimg/d1.original.fstypes cp /home/user/czimg/sda1.ntfs-ptcl-img.gz.aa /images/fogimg/d1p1.img cat /home/user/czimg/sda2.ntfs-ptcl-img.gz.* > /images/fogimg/d1p2.img
The last command will take the longest. If you want to see something happening, install pipe viewer and replace the last command with:
cat /home/user/czimg/sda2.ntfs-ptcl-img.gz.* | pv > /images/fogimg/d1p2.img
Have fun
-
@dolf Maybe, with this addition, I can create some code that can do this automagically? May not be a while but I think it could be very useful. Of course I don’t know if gunzip will work if say the CloneZilla image is not already compressed.
-
@Tom-Elliott Or if something besides beginner mode is used in CloneZilla
-
If the CloneZilla image is not compressed, it will not have the
.gz
extention (easy to check for), and the last command becomes:cat /home/user/czimg/sda2.ntfs-ptcl-img.* | pigz -stdout > /images/fogimg/d1p2.img
If CloneZilla was invoked using dd, we can concatenate and pipe to partclone:
cat /home/user/czimg/sda2.ntfs-dd-img.* | partclone.$fstype -fsck-src-part -c -s - -O - | pigz -c > /images/fogimg/d1p2.img
etc… There might be a few more cases to consider.