• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

dmraid and mdadm?

Scheduled Pinned Locked Moved Unsolved
FOG Problems
3
33
8.7k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • T
    Tom Elliott
    last edited by Dec 2, 2016, 10:32 PM

    To add on, the file that handles assembling of raid devices is: /etc/init.d/S99fog.

    Particularly lines 17 thru 20.

    Mind you we’re not starting dmraid, we’re actually using mdadm

    Maybe try a mechanism that can use mdadm to assemble the disks?

    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

    R 1 Reply Last reply Dec 2, 2016, 10:34 PM Reply Quote 0
    • R
      rbaldwin @Tom Elliott
      last edited by rbaldwin Dec 5, 2016, 8:52 AM Dec 2, 2016, 10:34 PM

      @Tom-Elliott

      I haven’t been able to make mdadm assemble the disk.

      For example:
      mdadm --assemble /dev/md0 /dev/sda /dev/sdb

      Complains that disk /dev/sda and /dev/sdb do not contain a superblock.

      T 1 Reply Last reply Dec 2, 2016, 10:35 PM Reply Quote 0
      • T
        Tom Elliott @rbaldwin
        last edited by Dec 2, 2016, 10:35 PM

        @rbaldwin They wouldn’t, you need to point at the partitions that are defined for them.

        (Of course that’s assuming there are partitions defined).

        What’s the output of fdisk -l for /dev/sda1 and /dev/sdb1?

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

        R 1 Reply Last reply Dec 2, 2016, 10:48 PM Reply Quote 0
        • T
          Tom Elliott
          last edited by Dec 2, 2016, 10:41 PM

          What about the output of mdadm --assemble --scan

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

          R T 3 Replies Last reply Dec 2, 2016, 10:50 PM Reply Quote 0
          • R
            rbaldwin @Tom Elliott
            last edited by rbaldwin Dec 2, 2016, 4:49 PM Dec 2, 2016, 10:48 PM

            @Tom-Elliott

            The raid array has a Windows 7 64bit installation on it and is defined as ~1TB

            Raid configuration in Bios:
            https://goo.gl/photos/iLZveDYK1vJ6Tijt6

            mdadm --examine on /dev/sda
            https://goo.gl/photos/9DTovJpVxsrLPApA9

            mdadm --examine on /dev/sdb
            https://goo.gl/photos/BY8ynhEzmsFNHtES6

            mdadm --examine --scan output:
            https://goo.gl/photos/Hq3CN682Hc3jshq1A

            As you can see mdadm can see the volume but for some reason doesn’t automatically assemble it. I don’t know eneough about how this works to say why. Just looking at dmraid because I’ve seen it work on the Ubuntu 16.04 live cd. Something I can’t say for mdadm so for on the P50. mdadm works fine on the Lenovo W530.

            fdisk -l output:
            https://goo.gl/photos/jMvtUtfyjsoyPZBH6

            1 Reply Last reply Reply Quote 0
            • R
              rbaldwin @Tom Elliott
              last edited by Dec 2, 2016, 10:50 PM

              @Tom-Elliott

              It’s in my last reply:
              https://goo.gl/photos/Hq3CN682Hc3jshq1A

              1 Reply Last reply Reply Quote 0
              • R
                rbaldwin @Tom Elliott
                last edited by Dec 2, 2016, 10:55 PM

                @Tom-Elliott

                Sorry just noticed you said mdadm --assemble --scan

                “no arrays found in configure file or automatically”

                1 Reply Last reply Reply Quote 0
                • T
                  Tom Elliott
                  last edited by Dec 2, 2016, 10:59 PM

                  http://unix.stackexchange.com/questions/273819/how-do-i-rebuild-create-assemble-an-imsm-raid-0-array-from-disk-images-instead

                  It appears you might need to specify a chunk size? I don’t know what that chunksize will be, but if you can test the ideas on the above link, and figure out, I can work to add a “chunk” size argument for the raid if all works.

                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                  Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                  Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                  R 1 Reply Last reply Dec 5, 2016, 2:37 PM Reply Quote 1
                  • R
                    rbaldwin @Tom Elliott
                    last edited by rbaldwin Dec 5, 2016, 8:57 AM Dec 5, 2016, 2:37 PM

                    @Tom-Elliott

                    I was able to build the array with the mdadm command shown here:
                    https://goo.gl/photos/7gtJf1hHVi4VAjj18

                    The build command seems to be the key here, as It works with and without the chunk command.
                    I haven’t mounted it yet to see if it’s readable, but this is progress from what I’ve seen so far. Not sure what to do now, since this works great in Debug mode, but wont be done automatically on fresh reboots to debug, capture or deploy.

                    1 Reply Last reply Reply Quote 0
                    • R
                      rbaldwin
                      last edited by Jan 5, 2017, 8:32 PM

                      Any pointers on this, I just need to be able to specify these commands to FOG when these hosts boot up for imaging?

                      T 1 Reply Last reply Jan 5, 2017, 9:42 PM Reply Quote 0
                      • T
                        Tom Elliott @rbaldwin
                        last edited by Jan 5, 2017, 9:42 PM

                        @rbaldwin The difference is your commands are implicit (with or without the chunk commands). The methods that are in use when you run just a “normal” runner, the code is greatly different.

                        Maybe it’s my unknowing how to do so. If you’d like, you can play with the code a bit (in a debug mode maybe?)

                        The information FOG currently uses (and works for the few other cases we have of people using RAID layout) is located here:
                        https://github.com/FOGProject/fogproject/blob/dev-branch/src/buildroot/package/fog/scripts/etc/init.d/S99fog#L17

                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                        T 1 Reply Last reply Jan 5, 2017, 11:17 PM Reply Quote 0
                        • T
                          Tom Elliott @Tom Elliott
                          last edited by Tom Elliott Jan 5, 2017, 5:19 PM Jan 5, 2017, 11:17 PM

                          Specifically, I’m hoping that you could run the mdadm --assemble --scan with a chunk size specified in the command line. If this successfully builds your imsm array, then I can adjust our arguments to accept the “chunk” size.

                          Perhaps:

                          mdadm --auto-detect --assemble --scan
                          

                          Of course I don’t have an array I could test this with, sorry.

                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                          T R 2 Replies Last reply Jan 5, 2017, 11:18 PM Reply Quote 0
                          • T
                            Tom Elliott @Tom Elliott
                            last edited by Jan 5, 2017, 11:18 PM

                            I’m working on thoughts so if my messages seem to drift around it’s probably just thinking a lot.

                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                            Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                            Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                            1 Reply Last reply Reply Quote 0
                            • T
                              Tom Elliott
                              last edited by Tom Elliott Jan 5, 2017, 5:45 PM Jan 5, 2017, 11:44 PM

                              Looking a bit more, I’ve added three hopeful bits for solving this.

                              First, the mdadm --auto-detect should be by itself as mdadm is then asking the kernel to activate the arrays. (Source: https://linux.die.net/man/8/mdadm)

                              So --assemble --scan is not “compatible” with the --auto-detect argument (mdadm --auto-detect --assemble --scan vs mdadm --auto-detect).

                              I’ve updated the code base in the init’s for handling this to try these three things:

                              mdadm --auto-detect
                              mdadm --assemble --scan
                              mdadm --incremental --run --scan
                              

                              If you jump on the dev-branch, it should update the inits for this new stuff. If these new changes still don’t work, please see if there’s a way we can do an auto assembly on the iMSM raid. If we cannot auto-assemble, it is extremely difficult (currently) to make a working method for you. I suppose I could add a “postinitload” scripts thing, similar to postdownload scripts. I’d really prefer seeing if the current code could handle this with these modifications or an auto assembly model over implicit command to generate the array.

                              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                              R 1 Reply Last reply Jan 6, 2017, 2:21 PM Reply Quote 0
                              • R
                                rbaldwin @Tom Elliott
                                last edited by rbaldwin Jan 6, 2017, 8:15 AM Jan 6, 2017, 2:05 PM

                                @Tom-Elliott said in dmraid and mdadm?:

                                Specifically, I’m hoping that you could run the mdadm --assemble --scan with a chunk size specified in the command line. If this successfully builds your imsm array, then I can adjust our arguments to accept the “chunk” size.

                                Perhaps:

                                mdadm --auto-detect --assemble --scan
                                

                                Of course I don’t have an array I could test this with, sorry.

                                –chunk is an invalid command on the --assemble command as is --auto-detect

                                1 Reply Last reply Reply Quote 0
                                • R
                                  rbaldwin @Tom Elliott
                                  last edited by Jan 6, 2017, 2:21 PM

                                  @Tom-Elliott said in dmraid and mdadm?:

                                  Looking a bit more, I’ve added three hopeful bits for solving this.

                                  First, the mdadm --auto-detect should be by itself as mdadm is then asking the kernel to activate the arrays. (Source: https://linux.die.net/man/8/mdadm)

                                  So --assemble --scan is not “compatible” with the --auto-detect argument (mdadm --auto-detect --assemble --scan vs mdadm --auto-detect).

                                  I’ve updated the code base in the init’s for handling this to try these three things:

                                  mdadm --auto-detect
                                  mdadm --assemble --scan
                                  mdadm --incremental --run --scan
                                  

                                  If you jump on the dev-branch, it should update the inits for this new stuff. If these new changes still don’t work, please see if there’s a way we can do an auto assembly on the iMSM raid. If we cannot auto-assemble, it is extremely difficult (currently) to make a working method for you. I suppose I could add a “postinitload” scripts thing, similar to postdownload scripts. I’d really prefer seeing if the current code could handle this with these modifications or an auto assembly model over implicit command to generate the array.

                                  I booted in debug mode and ran the following.
                                  mdadm --auto-detect
                                  mdadm --assemble --scan
                                  mdadm --incremental --run --scan

                                  /proc/mdstats shows 0 devices.

                                  mdadm --detail /dev/md0
                                  Shows inactive with 0 devices.

                                  A postinitload script would be great, if I could specify that per host. As I have many Lenovo W530/W520s with RAID that work just fine with the existing FOG RAID Configurations. It’s only the brand new EUFI Bios’ed Lenovo P50s that are having this issue.

                                  Is there perhaps a newer version of mdadm tools? Maybe these new Intel fake RAID controllers aren’t supported by the older version?

                                  T 1 Reply Last reply Jan 6, 2017, 5:55 PM Reply Quote 0
                                  • T
                                    Tom Elliott @rbaldwin
                                    last edited by Jan 6, 2017, 5:55 PM

                                    @rbaldwin But they are supported, they’re just to supporting auto assembly. You showed that they did get detected if you structured the call.

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      Tom Elliott @Tom Elliott
                                      last edited by Jan 6, 2017, 9:38 PM

                                      Maybe this could help?

                                      mdadm --assemble --scan --metadata=imsm
                                      mdadm --incremental --run --scan --metadata=imsm
                                      

                                      I am hoping to find something.

                                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                      1 Reply Last reply Reply Quote 0
                                      • Wayne WorkmanW
                                        Wayne Workman
                                        last edited by Jan 7, 2017, 7:25 AM

                                        Can we please summarize this thread in 1 or 2 posts please? It’s now 24 posts long and getting unwieldy. I want to help - probably as others do - but I’ve just read through 30 other posts and it’s now 1:30 AM.

                                        Please someone summarize this thread.

                                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                        Daily Clean Installation Results:
                                        https://fogtesting.fogproject.us/
                                        FOG Reporting:
                                        https://fog-external-reporting-results.fogproject.us/

                                        T 1 Reply Last reply Jan 8, 2017, 11:34 PM Reply Quote 0
                                        • T
                                          Tom Elliott @Wayne Workman
                                          last edited by Jan 8, 2017, 11:34 PM

                                          @Wayne-Workman I guess I don’t understand what you mean “summarize”.

                                          Essentially (if this is what you mean), @rbaldwin has a RAID setup that he’d like to capture/deploy images from/to. The common RAID setup is using a metadata of 0.90 (which I suspect is causing the inability to assemble the array currently).

                                          This thread is trying to get the RAID array in use to work with FOG. I’m trying to get it to figure out how to do this without necessarily having to know the drives and layouts just to perform the tasking.

                                          Capture is really the only thing that should need this except we capture partition data based on MBR/GPT layout on the disk. This will need to be rethought so images can be deployed to any disk regardless of if it is a RAID array. – This is for later though (more a todo than will be done right now).

                                          So far, however, we have not had any luck so many of the runs on here is just a collaborative effort to see if we can get it working automatically.

                                          All we’re trying to do right now is figure out a means to get this somewhat automated. If we cannot I might add a postinitload feature that will allow users to perform tasks after the init is loaded but before fog start performing its own tasks.

                                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                          R 1 Reply Last reply Jan 9, 2017, 2:56 PM Reply Quote 0
                                          • 1
                                          • 2
                                          • 1 / 2
                                          • First post
                                            Last post

                                          215

                                          Online

                                          12.1k

                                          Users

                                          17.3k

                                          Topics

                                          155.3k

                                          Posts
                                          Copyright © 2012-2024 FOG Project