• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

How to manually upload an existing image

Scheduled Pinned Locked Moved
General
4
15
4.7k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • D
    dolf
    last edited by Jul 13, 2016, 3:31 AM

    Say I have a few images made using CloneZilla. The folder structure looks like this:

    user@myserver:~/partimag$ tree -ash
    .
    ├── [4.0K]  image-name
    │   ├── [ 677]  blkdev.list
    │   ├── [ 249]  blkid.list
    │   ├── [6.3K]  clonezilla-img
    │   ├── [ 159]  dev-fs.list
    │   ├── [   4]  disk
    │   ├── [ 20K]  Info-dmi.txt
    │   ├── [ 15K]  Info-lshw.txt
    │   ├── [2.0K]  Info-lspci.txt
    │   ├── [ 216]  Info-packages.txt
    │   ├── [  90]  Info-saved-by-cmd.txt
    │   ├── [  10]  parts
    │   ├── [  33]  sda1.info
    │   ├── [8.8M]  sda1.ntfs-ptcl-img.gz.aa
    │   ├── [3.8G]  sda2.ntfs-ptcl-img.gz.aa
    │   ├── [3.8G]  sda2.ntfs-ptcl-img.gz.ab
    │   ├── [3.8G]  sda2.ntfs-ptcl-img.gz.ac
    │   ├── [3.8G]  sda2.ntfs-ptcl-img.gz.ad
    │   ├── [3.8G]  sda2.ntfs-ptcl-img.gz.ae
    │   ├── [3.8G]  sda2.ntfs-ptcl-img.gz.af
    │   ├── [3.8G]  sda2.ntfs-ptcl-img.gz.ag
    │   ├── [3.8G]  sda2.ntfs-ptcl-img.gz.ah
    │   ├── [1.5G]  sda2.ntfs-ptcl-img.gz.ai
    │   ├── [  37]  sda-chs.sf
    │   ├── [1024K]  sda-hidden-data-after-mbr
    │   ├── [ 512]  sda-mbr
    │   ├── [ 333]  sda-pt.parted
    │   ├── [ 295]  sda-pt.parted.compact
    │   └── [ 190]  sda-pt.sf
    etc...
    

    FOG Images, however, look like this:

    user@myserver:/images$ tree
    .
    ├── an-old-image
    │   ├── d1.fixed_size_partitions
    │   ├── d1.original.fstypes
    │   ├── d1.original.partitions
    │   ├── d1.original.swapuuids
    │   ├── rec.img.000
    │   └── sys.img.000
    ├── a-newer-image
    │   ├── d1.fixed_size_partitions
    │   ├── d1.mbr
    │   ├── d1.minimum.partitions
    │   ├── d1.original.fstypes
    │   ├── d1.original.swapuuids
    │   ├── d1p1.img
    │   ├── d1p2.img
    │   └── d1.partitions
    etc...
    

    Could I somehow import the CloneZilla images into FOG, without deploying and recapturing the image?

    1 Reply Last reply Reply Quote 1
    • W
      Wayne Workman
      last edited by Wayne Workman Jul 12, 2016, 9:51 PM Jul 13, 2016, 3:49 AM

      Oh boy… While I’m sure it’s possible, someone is going to have to become familiar with how CloneZilla stores images to do it. To be honest - it’d probably be easier to stick with FOG and get it working and spread what you learn so the Devs can consider it and so others can read it. But I’ll try to point you in the right direction.

      Just glancing at the files, the 3.8GB files are obviously gun-zipped chunks. Those will need decompressed into one big image file. You’ll need to mount that via NTFS-3G in a Linux system, and then manually run the Partclone command that would take the image and compress it. This of course is just for partitions 1 and 2.

      sda-mbr is obviously the MBR, and its 512 bytes. That can just be renamed to d1.mbr and then placed. The files parts sda1.info and parts sound important, you should post the contents of those.

      And, 1,024 bytes worth of info after the MBR but before partition 1 sounds interesting, the sda-hidden-data-after-mbr file. That could honestly be the missing piece for your other thread: https://forums.fogproject.org/topic/8059/pc-unbootable-after-capture-fails/14

      Additional reading on the subject of manually manipulating images for use with FOG: https://forums.fogproject.org/topic/7432/mount-and-extract-files-from-images?page=1

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
      Daily Clean Installation Results:
      https://fogtesting.fogproject.us/
      FOG Reporting:
      https://fog-external-reporting-results.fogproject.us/

      1 Reply Last reply Reply Quote 0
      • D
        dolf
        last edited by Jul 13, 2016, 4:02 AM

        CloneZilla files, excluding the large ones.

        W 1 Reply Last reply Jul 13, 2016, 4:21 AM Reply Quote 1
        • W
          Wayne Workman @dolf
          last edited by Wayne Workman Jul 12, 2016, 10:22 PM Jul 13, 2016, 4:21 AM

          @dolf Ok, I had a quick look through those - seems very simple, I like it.

          disk - appears to be a list of all disks. this one just says sda
          sda-pt.sf, sda-pt.parted.compact, and sda-pt.parted appear to be similar to an fdisk -l output.
          sda-mbr is clearly a copy of the mbr.
          sda-chs.sf is information about the disk’s cylinders, heads, and sectors.
          sda1.info is just saying that’s the boot partition.
          parts is a list of partitions. This one has just sda1 and sda2.
          info-saved-by-cmd.txt appears to be output from a command CZ uses.
          info-packages.txt is a list of exact packages that build of CZ used.
          the info files are info about the computer it was pulled from.
          dev-fs.list is a list of partitions and their format.
          clonezilla-img is literally the output that CZ gave during the image it looks.
          blkid.list is simply the output of blkid
          blkdev.list I assume is the output of a package called blkdev

          Really, to convert this I think all you need is the information from the sda-pt.sf file. This would be a non-resizable image in fog. Use an existing non-resizable image as a template to help guide you with putting the numbers in the right spots. Change sda-mbr to d1.mbr and then you need to uncompress and mount the image, and then capture+compress with partclone and place all files where they should be, assign permissions, and probably fail a few times before it’s right.

          Of course the extra info between the mbr and partitions is what’s really interesting. It could just be meaningless gibberish in unused space, but maybe not…

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
          Daily Clean Installation Results:
          https://fogtesting.fogproject.us/
          FOG Reporting:
          https://fog-external-reporting-results.fogproject.us/

          1 Reply Last reply Reply Quote 0
          • D
            dolf
            last edited by dolf Jul 12, 2016, 10:26 PM Jul 13, 2016, 4:25 AM

            WOW thanks for the advice. I’ll try that soon.

            I think the 1MB space between the partitions is just for alignment.

            W 1 Reply Last reply Jul 13, 2016, 4:35 AM Reply Quote 1
            • W
              Wayne Workman @dolf
              last edited by Jul 13, 2016, 4:35 AM

              @dolf I’d urge you to try a non-resizable image in fog first. See if it works. If all the computers are identical they will all have the same size HDD. It’ll either work or not. If it works, that’s how you do it for this model.

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
              Daily Clean Installation Results:
              https://fogtesting.fogproject.us/
              FOG Reporting:
              https://fog-external-reporting-results.fogproject.us/

              1 Reply Last reply Reply Quote 0
              • D
                dolf
                last edited by Jul 13, 2016, 5:15 AM

                OK so while the imaging runs, I’m reading code.

                CloneZilla saves the partition like this (from the log file clonezilla-img😞

                partclone.ntfs -z 10485760 -N -L /var/log/partclone.log -c -s /dev/sda2 --output - | pigz -c --fast -b 1024 -p 16 --rsyncable | split -a 2 -b 4096MB - /home/partimag/2016-07-12-21-img-eelabtest/sda2.ntfs-ptcl-img.gz. 2> /tmp/split_error.TMVF3J
                

                Fog does it like this (from savePartition() in src/buildroot/package/fog/scripts/usr/share/fog/lib/funcs.sh)

                partclone.$fstype -fsck-src-part -c -s $part -O $fifoname -N -f 1
                

                I’m not really sure how the FIFO buffer works, but it’s called /tmp/pigz1, so I guess it’s the same as piping the output of partclone through pigz, which is exactly what CloneZilla does. Therefore, maybe this would be sufficient to “convert from CloneZilla format to Fog format”:

                cat sda2.ntfs-ptcl-img.gz.* > d1p2.img
                

                In other words, no recompression is needed. Am I missing something here?

                1 Reply Last reply Reply Quote 3
                • D
                  dolf
                  last edited by Jul 13, 2016, 6:12 AM

                  I created a new image in the Fog web interface, copied the d1.* files from a very similar image, and replaced d1p*.img with concatenated versions of the CloneZilla images. Currently deploying, and partclone seems happy thus far. Just wondering whether the resize will work…

                  By the way:
                  Sorry for the delayed responses. It typically takes me 45 minutes to deploy a 65GB image. Without compression the shortest time should be around ((65×1024)MB ÷ (100÷8)MB/s) ÷ 60s = 88 minutes. There’s a 100MB switch somewhere between here and the server 😞

                  Q 1 Reply Last reply Jul 13, 2016, 2:26 PM Reply Quote 1
                  • D
                    dolf
                    last edited by Jul 13, 2016, 8:27 AM

                    It works! 🙂 The minimum partition size is probably wrong, but I will only deploy to larger drives in any case.

                    W 1 Reply Last reply Jul 13, 2016, 2:21 PM Reply Quote 2
                    • W
                      Wayne Workman @dolf
                      last edited by Jul 13, 2016, 2:21 PM

                      @dolf Wow awesome man, way to go! I had serious doubts about this being easy!

                      Can you please lay out details on all your steps to help people in the future?

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                      Daily Clean Installation Results:
                      https://fogtesting.fogproject.us/
                      FOG Reporting:
                      https://fog-external-reporting-results.fogproject.us/

                      1 Reply Last reply Reply Quote 0
                      • Q
                        Quazz Moderator @dolf
                        last edited by Jul 13, 2016, 2:26 PM

                        @dolf Could also be a CAT 5 cable somewhere.

                        1 Reply Last reply Reply Quote 0
                        • D
                          dolf
                          last edited by Jul 13, 2016, 10:47 PM

                          Feel free to put this in the wiki if you think it’s worthy.

                          I’m working with a Windows 7 installation, where sda1 is a small boot partition and sda2 is the large partition called C:.

                          1. Use GParted (from your favourite linux disc or GParted Live) to resize sda2 to a minimum.
                          2. Use CloneZilla to capture the disk. I used beginner mode with the savedisk option. I transferred it to /home/user/czimg/… on the FOG server over ssh, but you could use a USB HDD or any other method.
                          3. Create a new resizable image on the FOG web interface. Note the image location.
                          4. Create the location specified in the previous step, e.g. mkdir /images/fogimg
                          5. Do magic:
                          cp /home/user/czimg/sda-pt.sf /images/fogimg/d1.minimum.partitions
                          cp /home/user/czimg/sda-pt.sf /images/fogimg/d1.partitions
                          cp /home/user/czimg/sda-mbr /images/fogimg/d1.mbr
                          echo "1" > /images/fogimg/d1.fixed_size_partitions
                          echo "/dev/sda2 ntfs" > /images/fogimg/d1.original.fstypes
                          cp /home/user/czimg/sda1.ntfs-ptcl-img.gz.aa /images/fogimg/d1p1.img
                          cat /home/user/czimg/sda2.ntfs-ptcl-img.gz.* > /images/fogimg/d1p2.img
                          

                          The last command will take the longest. If you want to see something happening, install pipe viewer and replace the last command with:

                          cat /home/user/czimg/sda2.ntfs-ptcl-img.gz.* | pv > /images/fogimg/d1p2.img
                          

                          Have fun

                          T 1 Reply Last reply Jul 14, 2016, 12:23 AM Reply Quote 2
                          • T
                            Tom Elliott @dolf
                            last edited by Jul 14, 2016, 12:23 AM

                            @dolf Maybe, with this addition, I can create some code that can do this automagically? May not be a while but I think it could be very useful. Of course I don’t know if gunzip will work if say the CloneZilla image is not already compressed.

                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                            Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                            Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                            W 1 Reply Last reply Jul 14, 2016, 12:26 AM Reply Quote 0
                            • W
                              Wayne Workman @Tom Elliott
                              last edited by Jul 14, 2016, 12:26 AM

                              @Tom-Elliott Or if something besides beginner mode is used in CloneZilla

                              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                              Daily Clean Installation Results:
                              https://fogtesting.fogproject.us/
                              FOG Reporting:
                              https://fog-external-reporting-results.fogproject.us/

                              1 Reply Last reply Reply Quote 0
                              • D
                                dolf
                                last edited by Jul 14, 2016, 6:18 AM

                                If the CloneZilla image is not compressed, it will not have the .gz extention (easy to check for), and the last command becomes:

                                cat /home/user/czimg/sda2.ntfs-ptcl-img.* | pigz -stdout > /images/fogimg/d1p2.img
                                

                                If CloneZilla was invoked using dd, we can concatenate and pipe to partclone:

                                cat /home/user/czimg/sda2.ntfs-dd-img.* | partclone.$fstype -fsck-src-part -c -s - -O - | pigz -c > /images/fogimg/d1p2.img
                                

                                etc… There might be a few more cases to consider.

                                1 Reply Last reply Reply Quote 1
                                • 1 / 1
                                1 / 1
                                • First post
                                  13/15
                                  Last post

                                199

                                Online

                                12.0k

                                Users

                                17.3k

                                Topics

                                155.2k

                                Posts
                                Copyright © 2012-2024 FOG Project