• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

New FOG Testing dashboard. Feedback?

Scheduled Pinned Locked Moved
General
3
13
1.2k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • W
    Wayne Workman
    last edited by Wayne Workman Dec 15, 2018, 5:34 PM Dec 15, 2018, 11:03 PM

    This link is the original fog testing dashboard.
    This dashboard was created to help identify problems with the FOG installation process early on, so they can be fixed before users experience them. This dashboard has been generally UP for about two years now. It’s served the FOG Project well.

    However, the old dashboard was written mostly in BASH and is a mess. It requires custom setup, custom snapshots, hand-spun virtual machines, and a tremendous amount of compute resources and time. While the code that this dashboard uses is completely open source and available, due to the complex setup, nobody has bothered using it.

    Thank you, old dashboard, You’re retirement party is coming up soon.

    Hello new dashboard!

    The new dashboard has several significant improvements over the old one. Notably:

    • Mostly written in Python.
    • 100% AWS based.
    • 100% of the infrastructure is built via Terraform.
    • Includes Red Hat Enterprise Linux 7 testing.
    • Orders of magnitude faster to run.
    • Makes logs available regardless of success status.
    • New logs, and output - such as stdout from the fog installer, patching stdout, and php-fpm logs.

    The new code base has been made available for all here and includes instructions for you to set this up for yourself, should you decide to run the FOG Installer tests yourself.

    Features that existed in the old dashboard but were done away with (for now) are:

    • Streak counts
    • Log history.
    • FogProject slack channel posts.
    • Older OSs.

    I plan to make testing history available in the future.

    For a while, I plan to run the old tests and new tests in parallel until I’m sure the new tests are stable, bug free, and ready to be relied on. Then, the old tests will be shut down for good.

    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
    Daily Clean Installation Results:
    https://fogtesting.fogproject.us/
    FOG Reporting:
    https://fog-external-reporting-results.fogproject.us/

    1 Reply Last reply Reply Quote 2
    • S
      Sebastian Roth Moderator
      last edited by Dec 16, 2018, 12:33 PM

      @Wayne-Workman I really like the new one with all the log files available!

      Would it be possible to keep log files from the last successful build? I mean, if it fails we have the logs form that. But what was it like when it worked the last time? Simply don’t clean up the log files form earlier as long as the installer fails. But accessing the old logs might not be as easy/nice.

      By the way, would you mind starting a run on the new dashboard whenever you have time to. I hopefully fixed the issue for redhat based systems in dev-branch which I broke yesterday.

      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

      W 1 Reply Last reply Dec 16, 2018, 2:34 PM Reply Quote 0
      • W
        Wayne Workman @Sebastian Roth
        last edited by Dec 16, 2018, 2:34 PM

        @Sebastian-Roth Going, should be done in around 20 minutes.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
        Daily Clean Installation Results:
        https://fogtesting.fogproject.us/
        FOG Reporting:
        https://fog-external-reporting-results.fogproject.us/

        1 Reply Last reply Reply Quote 0
        • S
          Sebastian Roth Moderator
          last edited by Sebastian Roth Dec 16, 2018, 9:10 AM Dec 16, 2018, 3:09 PM

          @Wayne-Workman Ok, thanks! CentOS and RHEL were fixed already. I just pushed another fix for Fedora and hopefully Arch. Mind running it again?

          By the way, do you know why CentOS takes usually double the time than most others?

          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

          W 1 Reply Last reply Dec 16, 2018, 3:14 PM Reply Quote 0
          • W
            Wayne Workman @Sebastian Roth
            last edited by Wayne Workman Dec 16, 2018, 9:15 AM Dec 16, 2018, 3:14 PM

            @Sebastian-Roth 2nd round going.

            For CentOS taking a long time is probably two reasons.

            • The installer does tons of stuff to get FOG working on CentOS 7.
            • The CentOS 7 instance type is t2.micro. All others are t3.micro. The t3 series is newer hardware, more ram & cores, and is cheaper to run. I had to use a t2.micro with CentOS 7 because CentOS’s official AMIs do not support the advanced networking hardware that the t3s use.

            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
            Daily Clean Installation Results:
            https://fogtesting.fogproject.us/
            FOG Reporting:
            https://fog-external-reporting-results.fogproject.us/

            1 Reply Last reply Reply Quote 1
            • W
              Wayne Workman
              last edited by Dec 16, 2018, 3:42 PM

              I think the way to provide the last successful logs is links, and just keep all these dashboards, and to use html includes with JS.

              This dashboard is being hosted from an s3 bucket and has no ability to do server side processing. Thus, it’s all gotta be in links or JS.

              https://www.w3schools.com/howto/howto_html_include.asp
              I’m thinking another column between Install Duration and Install Output that links to the dashboard where that particular test was last successful. And overall, index.html will just load up the latest dashboard using JS.

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
              Daily Clean Installation Results:
              https://fogtesting.fogproject.us/
              FOG Reporting:
              https://fog-external-reporting-results.fogproject.us/

              1 Reply Last reply Reply Quote 0
              • W
                Wayne Workman
                last edited by Dec 27, 2018, 12:28 AM

                I’ve shut down the old tests, and switched the new tests to use fogtesting.theworkmans.us

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                Daily Clean Installation Results:
                https://fogtesting.fogproject.us/
                FOG Reporting:
                https://fog-external-reporting-results.fogproject.us/

                F 1 Reply Last reply Dec 31, 2018, 5:19 PM Reply Quote 2
                • F
                  fry_p Moderator @Wayne Workman
                  last edited by Dec 31, 2018, 5:19 PM

                  @Wayne-Workman said in New FOG Testing dashboard. Feedback?:

                  I’ve shut down the old tests, and switched the new tests to use fogtesting.theworkmans.us

                  Well done man! I’m going to miss the successful patch streaks just because that was fun to look at. I understand why its gone though. I am probably due for an update to 1.5.5 from 1.5.4 and this is a great resource.

                  Like open source community computing? Why not do it for a good cause?
                  Use your computer/server for humanitarian projects when it is idle!
                  https://join.worldcommunitygrid.org?recruiterId=1026912

                  1 Reply Last reply Reply Quote 1
                  • S
                    Sebastian Roth Moderator
                    last edited by Jan 15, 2019, 1:59 PM

                    @Wayne-Workman Tests for Arch seem to fail when installing package php-gd dependencies:

                    error: libtiff: signature from "Levente Polyak (anthraxx) <levente@leventepolyak.net>" is unknown trust
                    :: File /var/cache/pacman/pkg/libtiff-4.0.10-1-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
                    Do you want to delete it? [Y/n] error: failed to commit transaction (invalid or corrupted package)
                    

                    And also vsftpd:

                    error: vsftpd: signature from "Levente Polyak (anthraxx) <levente@leventepolyak.net>" is unknown trust
                    :: File /var/cache/pacman/pkg/vsftpd-3.0.3-4-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
                    Do you want to delete it? [Y/n] error: failed to commit transaction (invalid or corrupted package (PGP signature))
                    

                    No idea why. Maybe we just wait and see what it looks like tomorrow.

                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                    W 1 Reply Last reply Jan 16, 2019, 12:20 AM Reply Quote 0
                    • W
                      Wayne Workman @Sebastian Roth
                      last edited by Wayne Workman Jan 15, 2019, 6:20 PM Jan 16, 2019, 12:20 AM

                      @Sebastian-Roth I emailed that guy’s address in the error and asked him what’s wrong.

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                      Daily Clean Installation Results:
                      https://fogtesting.fogproject.us/
                      FOG Reporting:
                      https://fog-external-reporting-results.fogproject.us/

                      1 Reply Last reply Reply Quote 1
                      • S
                        Sebastian Roth Moderator
                        last edited by Jan 18, 2019, 12:18 PM

                        @Wayne-Workman As it is still an issue I did a quick google foo and came up with this: https://wiki.archlinux.org/index.php/Pacman#Signature_from_.22User_.3Cemail.40example.org.3E.22_is_unknown_trust.2C_installation_failed

                        This is known and has been mentioned in forums many times:
                        https://bbs.archlinux.org/viewtopic.php?id=221435
                        https://bbs.archlinux.org/viewtopic.php?id=231555
                        https://bbs.archlinux.org/viewtopic.php?id=223455
                        https://bbs.archlinux.org/viewtopic.php?id=221641

                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                        W 1 Reply Last reply Jan 18, 2019, 1:25 PM Reply Quote 1
                        • W
                          Wayne Workman @Sebastian Roth
                          last edited by Jan 18, 2019, 1:25 PM

                          @Sebastian-Roth I’ll add the command pacman -Sy archlinux-keyring && pacman -Syyu into the mix soon, I’m sure it’ll resolve the issue.

                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                          Daily Clean Installation Results:
                          https://fogtesting.fogproject.us/
                          FOG Reporting:
                          https://fog-external-reporting-results.fogproject.us/

                          1 Reply Last reply Reply Quote 0
                          • W
                            Wayne Workman
                            last edited by Jan 21, 2019, 6:23 AM

                            I made the changes and did a test run, dev-branch completely passed.

                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                            Daily Clean Installation Results:
                            https://fogtesting.fogproject.us/
                            FOG Reporting:
                            https://fog-external-reporting-results.fogproject.us/

                            1 Reply Last reply Reply Quote 2
                            • 1 / 1
                            • First post
                              Last post

                            201

                            Online

                            12.0k

                            Users

                            17.3k

                            Topics

                            155.2k

                            Posts
                            Copyright © 2012-2024 FOG Project