• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Supporting LVM

    Scheduled Pinned Locked Moved
    General
    2
    6
    1.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Wayne WorkmanW
      Wayne Workman
      last edited by

      This thread is to capture discussion around LVM support and related items.

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
      Daily Clean Installation Results:
      https://fogtesting.fogproject.us/
      FOG Reporting:
      https://fog-external-reporting-results.fogproject.us/

      1 Reply Last reply Reply Quote 0
      • Wayne WorkmanW
        Wayne Workman
        last edited by

        My initial thoughts on supporting LVM was bash scripts. Then I started thinking about the tooling that would be involved. Intensive use of many bash parsing techniques that we Linux users know all too well. I thought - geeze why don’t these tools just output JSON for us so we don’t have to do bash-wizardry to get the values?

        So here’s where my head-space is now: Fork these partition/LVM information gathering tools and add JSON output options to them. This will be a challenge for me personally, as the tools such as fdisk and lvm are likely written in either C or C++. I want to add something like a --output-as-json argument or similar. This is to eliminate all the complex parsing we currently do. Then, something like JQ could be used to easily pull values from the JSON as needed.

        Second, how the disk information for an image is stored. We currently use text files stored with the images in their respective directories on the FOG server. Those files have a structure, yes. But I think JSON would be better.

        Considering handling legacy images (non-json) I’ve not thought too much on this yet but my initial thoughts is to convert all the text files during a fog install upgrade. Not getting rid of them, but putting their converted counterparts in with the images; leaving the old files in-place. And the upgraded fog system just wouldn’t use the old files.

        Looking for some thoughts on this approach before I dive in deep.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
        Daily Clean Installation Results:
        https://fogtesting.fogproject.us/
        FOG Reporting:
        https://fog-external-reporting-results.fogproject.us/

        1 Reply Last reply Reply Quote 0
        • S
          Sebastian Roth Moderator
          last edited by

          @Wayne-Workman Sorry I have not answered this yet. It’s been very busy over the last weeks and I hardly get to FOG even now.

          So here’s where my head-space is now: Fork these partition/LVM information gathering tools and add JSON output options to them. This will be a challenge for me personally, as the tools such as fdisk and lvm are likely written in either C or C++. I want to add something like a --output-as-json argument or similar. This is to eliminate all the complex parsing we currently do. Then, something like JQ could be used to easily pull values from the JSON as needed.

          While I like the idea in general I think it’s a hell of a job to do for several different tools. I could be wrong but that is what my guts tell me. This is me digging through C/C++ code and also a tiny bit of coding on and off for at lease 15 years.

          Considering handling legacy images (non-json) I’ve not thought too much on this yet but my initial thoughts is to convert all the text files during a fog install upgrade. Not getting rid of them, but putting their converted counterparts in with the images; leaving the old files in-place. And the upgraded fog system just wouldn’t use the old files.

          Sounds like a good strategy to convert files and keep the original ones as well.

          All in all this is a huge project you are aiming for. Before you get into it we might think about different options we have and then compare pros and cons.

          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

          Wayne WorkmanW 1 Reply Last reply Reply Quote 1
          • Wayne WorkmanW
            Wayne Workman @Sebastian Roth
            last edited by Wayne Workman

            @sebastian-roth I’ve been thinking on your comments since you’ve posted them.

            I have another idea that would achieve the same ends. A python wrapper. I really, really want to avoid the complexities involved with BASH based parsing. I’m just completely turned off by it, and not interested. Really, I believe firmly all CLI tools ought to have a JSON output option. Seriously! (Maybe I should start a movement?)

            Though, the tools do not currently output JSON. I’m skilled with Python and could write a wrapper for these tools, to convert the output to a nice JSON format. Thinking we can call a python script that collects all necessary partitioning information, and spits it all out via stdout in JSON format.

            The one hitch - can we get python3 into FOS? Maybe it’s already there? Haven’t checked.

            I don’t want to abandon this effort. I’m just looking for a way forward that is both attainable as well as not miserable.

            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
            Daily Clean Installation Results:
            https://fogtesting.fogproject.us/
            FOG Reporting:
            https://fog-external-reporting-results.fogproject.us/

            1 Reply Last reply Reply Quote 0
            • S
              Sebastian Roth Moderator
              last edited by

              @wayne-workman While a wrapper is adding complexity and therefore failures I still think we should look into this. Sounds strange but I am prone to BASH parsing violence. Though I totally understand this is not sexy and causes issues too.

              Really, I believe firmly all CLI tools ought to have a JSON output option. Seriously! (Maybe I should start a movement?)

              That would be really cool! Hope my last message didn’t turn you down. You are more than welcome to just pick one tool and give it a try. Start coding and find out if you can get this pushed into the upstream code as well. It’s always a win for the OSS community and not a waste of time! After this first one there is a good chance we/you know if it’s possible to do for all the tools needed.

              The one hitch - can we get python3 into FOS? Maybe it’s already there? Haven’t checked.

              Don’t think we have it included yet (config reference. But that’s no big issue. We can add that and build new inits. Just let me know if you want to go this route or try the other one first?

              On the downside Python(3) will surely add to the size of the inits and slow down the PXE boot process a bit. Not sure how much yet. Though we’ll do it if that’s the cost for adding LVM support!

              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

              1 Reply Last reply Reply Quote 0
              • Wayne WorkmanW
                Wayne Workman
                last edited by

                Another thing I just now thought of - versioning the partition file we keep with the images. We can start out with v1.0.0. And if we change how we store that file, we simply increment the version. This would give us some more flexibility in regard to changes.

                Thinking further, the scripts that collect the physical partitions are fine, we can just add JSON output for those.

                And really, the LVM describing tools are what needs JSON output, so the effort there is targeted to just those.

                I think I’m going to take a swing at adding JSON to an LVM describing tool (there are several).

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                Daily Clean Installation Results:
                https://fogtesting.fogproject.us/
                FOG Reporting:
                https://fog-external-reporting-results.fogproject.us/

                1 Reply Last reply Reply Quote 1
                • 1 / 1
                • First post
                  Last post

                154

                Online

                12.0k

                Users

                17.3k

                Topics

                155.2k

                Posts
                Copyright © 2012-2024 FOG Project