• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Development FOG not capturing image - PartClone update

    Scheduled Pinned Locked Moved Solved
    FOG Problems
    5
    36
    4.9k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      ty900000 @Tom Elliott
      last edited by

      @Tom-Elliott

      I stepped through everything until it halted. Pressing [Enter] here doesn’t do anything.

      FOG.PNG

      1 Reply Last reply Reply Quote 0
      • S
        Sebastian Roth Moderator
        last edited by Sebastian Roth

        @Tom-Elliott I really wonder why we don’t see other people report this error. Were you actually able to replicate this? Maybe this is just some RAM issue that causes binaries to fail on this particular machine!?

        By the way, @ty900000 would you mind opening a new topic for this? Better to keep things sorted. I can move all the related messages over…

        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

        T Tom ElliottT 2 Replies Last reply Reply Quote 0
        • Tom ElliottT
          Tom Elliott @Sebastian Roth
          last edited by

          @Sebastian-Roth @ty900000

          Would you mind trying the latest inits from: https://dev.fogproject.org/job/fos/job/master/lastSuccessfulBuild/

          The init.xz and init_32.xz should be good.

          Essentially I’m having a check on the partclone to be used and removing a couple of arguments as they are not built during the configuration and build of partclone.

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

          T 1 Reply Last reply Reply Quote 0
          • T
            ty900000 @Tom Elliott
            last edited by

            @Tom-Elliott

            Yes! It worked perfectly. I’ve tested it a bunch of times and it works great. I do get this output after one of the partitions. It doesn’t affect anything it seems, but I’ve just noticed it.

            FOG.PNG

            1 Reply Last reply Reply Quote 0
            • S
              Sebastian Roth Moderator
              last edited by

              @Tom-Elliott Are you able to replicate the issue as seen in the pictures?
              @ty900000 Does this happen on several machines? All the same model or different ones?

              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

              T 1 Reply Last reply Reply Quote 0
              • T
                ty900000 @Sebastian Roth
                last edited by

                @Sebastian-Roth

                To start, I am using Hyper-V for everything. Yes, I do get that above image when I try to capture other images - either Windows or Linux. When I attempt to deploy the Windows image (the original image I’ve been trying to take), I get this error. But it does seem to complete. It does something similar for the the Linux image.

                FOG.PNG

                FOG.PNG

                Tom ElliottT 1 Reply Last reply Reply Quote 0
                • Tom ElliottT
                  Tom Elliott @ty900000
                  last edited by

                  @ty900000 @Sebastian-Roth
                  I haven’t replicated, but to be fair I also haven’t watched that closely. We did image one machine yesterday and all seemed fine.

                  Looking at my images folder, however, I do notice that I’m missing the “imager” partition from my image. Luckily I had another image of the machine that did have the missing partition.

                  I pushed another fix and believe the issue was as @Quazz noted is the -c argument was missing. Strange as that is, as the -c argument doesn’t appear to be a part of the spec list (unless somebody already added that to the patch for partclone and I didn’t know it?)

                  This will take a while to build of course as I only just pushed it.

                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                  Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                  Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                  S 1 Reply Last reply Reply Quote 0
                  • S
                    Sebastian Roth Moderator
                    last edited by

                    @Tom-Elliott said in Development FOG not capturing image - PartClone update:

                    Strange as that is, as the -c argument doesn’t appear to be a part of the spec list

                    I think the -c is important to make partclone.imager actually use the parclone image format.

                    I haven’t replicated.

                    My guess is that this is something specific to Hyper-V or maybe even just @ty900000’s setup. Not saying we shouldn’t try to figure this out and eventually fix if it’s in the inits. My feeling is that this is not about partclone command line parameters or anything.

                    @ty900000 Please do me a favor and play with the the image’s setting Image Manager. Try Partclone Zstd if you have used Gzip since and even more so try out Partclone Uncompressed! Capture the image with these changed settings once more and let us know if it makes any difference.

                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                    T 1 Reply Last reply Reply Quote 0
                    • S
                      shruggy @Tom Elliott
                      last edited by

                      @Tom-Elliott said in Development FOG not capturing image - PartClone update:

                      I pushed another fix and believe the issue was as @Quazz noted is the -c argument was missing. Strange as that is, as the -c argument doesn’t appear to be a part of the spec list (unless somebody already added that to the patch for partclone and I didn’t know it?)

                      My understanding of it is this:

                      • images captured with partclone.imager -c need to be restored with partclone.restore, raw images captured with partclone.dd or with partclone.imager without -c need to be restored with partclone.dd
                      • the options for partclone.imager are basically the same as for any partclone.$ftype (including -c and -a), it’s partclone.dd that is a special case. The symbol DD only pertains to partclone.dd, the symbol IMG is specifically defined in Makefile for partclone.imager, but never gets used in upstream Partclone code.
                      Q 1 Reply Last reply Reply Quote 0
                      • T
                        ty900000 @Sebastian Roth
                        last edited by

                        @Sebastian-Roth

                        I did test with several of the other compression types. As far as I can figure the Partclone Uncompressed is the only one that throws the above error with the 1 minute time out. And I also noticed it is not capturing sda3 - which is the raw partition on my particular Windows image. I think when it gets done with sda2 and attempts to move to sda3, it throws the Usage: unset_name [OPTIONS] error from farther above.

                        The only reason I am using Uncompressed is because the Gzip and Zstd image captures with 6 compression were slower than I remember them being in older (much older) versions of FOG - about 3 GB/min capture and deploy whereas the uncompressed was 6+GB/min, which I know is about line speed. We had a server at my last job and we were getting 6+GB/min when we captured and deployed a machine with Zstd level 6 compression. I need to do some testing tomorrow with a physical machine capture and deploy to see what’s going on with my network…

                        1 Reply Last reply Reply Quote 0
                        • T
                          ty900000
                          last edited by ty900000

                          I pulled the latest init just now, too. Using both uncompressed and Zstd

                          FOG.PNG

                          1 Reply Last reply Reply Quote 0
                          • Q
                            Quazz Moderator @shruggy
                            last edited by

                            @shruggy said in Development FOG not capturing image - PartClone update:

                            @Tom-Elliott said in Development FOG not capturing image - PartClone update:

                            I pushed another fix and believe the issue was as @Quazz noted is the -c argument was missing. Strange as that is, as the -c argument doesn’t appear to be a part of the spec list (unless somebody already added that to the patch for partclone and I didn’t know it?)

                            My understanding of it is this:

                            • images captured with partclone.imager -c need to be restored with partclone.restore, raw images captured with partclone.dd or with partclone.imager without -c need to be restored with partclone.dd
                            • the options for partclone.imager are basically the same as for any partclone.$ftype (including -c and -a), it’s partclone.dd that is a special case. The symbol DD only pertains to partclone.dd, the symbol IMG is specifically defined in Makefile for partclone.imager, but never gets used in upstream Partclone code.

                            Adding on to this, partclone.imager is generated using partclone.dd source files.

                            As per the makefile: https://github.com/Thomas-Tsai/partclone/blob/master/src/Makefile.am#L56

                            Making this commit: https://github.com/Thomas-Tsai/partclone/commit/2d1ee7c94139f390453ea8cb8675f67173bde33b problematic.

                            The special code for RAW is then locked to partclone.dd while partclone.imager is stuck using the “normal” code, which doesn’t work properly for its scenario.

                            Restoring the old code for partclone.imager should work. (there is no option to change the if condition main.c since there is no opt for imager…)

                            The stuff in https://forums.fogproject.org/topic/14078/1-5-7-89-partclone-doesn-t-capture-an-image-in-dd-mode-wrong-options-in-fog-upload/17 should work.

                            We just need to create a patch for it imo.

                            1 Reply Last reply Reply Quote 0
                            • S
                              shruggy
                              last edited by

                              @ty900000 Could you please try my init.xz with patched Partclone and say if it works for you?

                              T 1 Reply Last reply Reply Quote 0
                              • T
                                ty900000 @shruggy
                                last edited by

                                @shruggy

                                That seems to work better. It is pull the ‘raw’ partition 3 and works when using uncompressed capture and deploy, Zstd compression 6 worked well too. I did see a few errors that didn’t seem to affect anything. The first one was after imaging with complete and the second image was before it started.

                                FOG.PNG

                                FOG2.PNG

                                1 Reply Last reply Reply Quote 0
                                • S
                                  shruggy
                                  last edited by shruggy

                                  @Quazz has just fixed it in the source.

                                  And I suppose the errors won’t show up if you set the image type to “Multiple Partition Image - Single Disk (Not Resizable)”, since the awk script gets only invoked during partition resizing.

                                  T 1 Reply Last reply Reply Quote 0
                                  • T
                                    ty900000 @shruggy
                                    last edited by

                                    @shruggy

                                    Okay, cool. I’ll wait for the build to complete and then pull it done for one more test. Thanks!

                                    Q 1 Reply Last reply Reply Quote 0
                                    • Q
                                      Quazz Moderator @ty900000
                                      last edited by

                                      @ty900000 That build does not yet include the partclone fixes, though. But we are getting close to that.

                                      1 Reply Last reply Reply Quote 1
                                      • S
                                        Sebastian Roth Moderator
                                        last edited by

                                        @ty900000 said in Development FOG not capturing image - PartClone update:

                                        I did test with several of the other compression types. As far as I can figure the Partclone Uncompressed is the only one that throws the above error with the 1 minute time out.

                                        and…

                                        I pulled the latest init just now, too. Using both uncompressed and Zstd
                                        … picture with segmentation fault …

                                        Those two answers don’t add up for me. Can you please do a simple test with the different compressions (none, Gzip, Zstd) and post which error exact you get for each.

                                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                        T 1 Reply Last reply Reply Quote 0
                                        • T
                                          ty900000 @Sebastian Roth
                                          last edited by

                                          @Sebastian-Roth

                                          My apologies; I know my previous message was confusing.

                                          A few days ago, the init from Jenkins and using Uncompressed would throw the error when it deployed - the image above about /dev/sda3. A few hours after that a new init was pushed to Jenkins that I tried. That one reintroduced the segmentation error during image capture, not matter what compression I tried.

                                          I currently am using the init @shruggy suggested yesterday and that one seems to have fixed everything for me during image capture and deployment.

                                          I haven’t pulled the latest init from the Jenkins. Would you like me to do that?

                                          Thanks everyone!

                                          1 Reply Last reply Reply Quote 0
                                          • S
                                            Sebastian Roth Moderator
                                            last edited by

                                            @ty900000 said:

                                            A few days ago, the init from Jenkins and using Uncompressed would throw the error when it deployed - the image above about /dev/sda3.

                                            I guess you mean this error: alt text

                                            That would mean the capture did run fine and partclone did not segfault when capturing. Though we probably can’t figure out which version it was you were using back then as too many things were tried in a rush. Nevermind.

                                            I currently am using the init @shruggy suggested yesterday …

                                            @shruggy Would you mind telling us which version of partclone this is?

                                            Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                            Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                            T S 2 Replies Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 2 / 2
                                            • First post
                                              Last post

                                            298

                                            Online

                                            12.0k

                                            Users

                                            17.3k

                                            Topics

                                            155.2k

                                            Posts
                                            Copyright © 2012-2024 FOG Project