New FOG Testing dashboard. Feedback?

[Wayne Workman]

This link is the original fog testing dashboard.
This dashboard was created to help identify problems with the FOG installation process early on, so they can be fixed before users experience them. This dashboard has been generally UP for about two years now. It’s served the FOG Project well.

However, the old dashboard was written mostly in BASH and is a mess. It requires custom setup, custom snapshots, hand-spun virtual machines, and a tremendous amount of compute resources and time. While the code that this dashboard uses is completely open source and available, due to the complex setup, nobody has bothered using it.

Thank you, old dashboard, You’re retirement party is coming up soon.

Hello new dashboard!

The new dashboard has several significant improvements over the old one. Notably:

• Mostly written in Python.
• 100% AWS based.
• 100% of the infrastructure is built via Terraform.
• Includes Red Hat Enterprise Linux 7 testing.
• Orders of magnitude faster to run.
• Makes logs available regardless of success status.
• New logs, and output - such as stdout from the fog installer, patching stdout, and php-fpm logs.

The new code base has been made available for all here and includes instructions for you to set this up for yourself, should you decide to run the FOG Installer tests yourself.

Features that existed in the old dashboard but were done away with (for now) are:

• Streak counts
• Log history.
• FogProject slack channel posts.
• Older OSs.

I plan to make testing history available in the future.

For a while, I plan to run the old tests and new tests in parallel until I’m sure the new tests are stable, bug free, and ready to be relied on. Then, the old tests will be shut down for good.

[Wayne Workman]

I made the changes and did a test run, dev-branch completely passed.

[Wayne Workman]

@Sebastian-Roth I’ll add the command pacman -Sy archlinux-keyring && pacman -Syyu into the mix soon, I’m sure it’ll resolve the issue.

[Sebastian Roth]

@Wayne-Workman As it is still an issue I did a quick google foo and came up with this: https://wiki.archlinux.org/index.php/Pacman#Signature_from_.22User_.3Cemail.40example.org.3E.22_is_unknown_trust.2C_installation_failed

This is known and has been mentioned in forums many times:
https://bbs.archlinux.org/viewtopic.php?id=221435
https://bbs.archlinux.org/viewtopic.php?id=231555
https://bbs.archlinux.org/viewtopic.php?id=223455
https://bbs.archlinux.org/viewtopic.php?id=221641

[Wayne Workman]

@Sebastian-Roth I emailed that guy’s address in the error and asked him what’s wrong.

[Sebastian Roth]

@Wayne-Workman Tests for Arch seem to fail when installing package php-gd dependencies:

error: libtiff: signature from "Levente Polyak (anthraxx) <levente@leventepolyak.net>" is unknown trust
:: File /var/cache/pacman/pkg/libtiff-4.0.10-1-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] error: failed to commit transaction (invalid or corrupted package)

And also vsftpd:

error: vsftpd: signature from "Levente Polyak (anthraxx) <levente@leventepolyak.net>" is unknown trust
:: File /var/cache/pacman/pkg/vsftpd-3.0.3-4-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] error: failed to commit transaction (invalid or corrupted package (PGP signature))

No idea why. Maybe we just wait and see what it looks like tomorrow.

[fry_p]

@Wayne-Workman said in New FOG Testing dashboard. Feedback?:

I’ve shut down the old tests, and switched the new tests to use fogtesting.theworkmans.us

Well done man! I’m going to miss the successful patch streaks just because that was fun to look at. I understand why its gone though. I am probably due for an update to 1.5.5 from 1.5.4 and this is a great resource.

[Wayne Workman]

I’ve shut down the old tests, and switched the new tests to use fogtesting.theworkmans.us

[Wayne Workman]

I think the way to provide the last successful logs is links, and just keep all these dashboards, and to use html includes with JS.

This dashboard is being hosted from an s3 bucket and has no ability to do server side processing. Thus, it’s all gotta be in links or JS.

https://www.w3schools.com/howto/howto_html_include.asp
I’m thinking another column between Install Duration and Install Output that links to the dashboard where that particular test was last successful. And overall, index.html will just load up the latest dashboard using JS.

[Wayne Workman]

@Sebastian-Roth 2nd round going.

For CentOS taking a long time is probably two reasons.

• The installer does tons of stuff to get FOG working on CentOS 7.
• The CentOS 7 instance type is t2.micro. All others are t3.micro. The t3 series is newer hardware, more ram & cores, and is cheaper to run. I had to use a t2.micro with CentOS 7 because CentOS’s official AMIs do not support the advanced networking hardware that the t3s use.

[Sebastian Roth]

@Wayne-Workman Ok, thanks! CentOS and RHEL were fixed already. I just pushed another fix for Fedora and hopefully Arch. Mind running it again?

By the way, do you know why CentOS takes usually double the time than most others?

[Wayne Workman]

@Sebastian-Roth Going, should be done in around 20 minutes.

[Sebastian Roth]

@Wayne-Workman I really like the new one with all the log files available!

Would it be possible to keep log files from the last successful build? I mean, if it fails we have the logs form that. But what was it like when it worked the last time? Simply don’t clean up the log files form earlier as long as the installer fails. But accessing the old logs might not be as easy/nice.

By the way, would you mind starting a run on the new dashboard whenever you have time to. I hopefully fixed the issue for redhat based systems in dev-branch which I broke yesterday.