Fog Installer - Distro check

[Wayne Workman]

Arch bombed out this morning on all branches - something to do with NFS. This is typical for Arch - one day it’ll just bomb out hard across the board on the same item, a couple days later it’s fixed. Once it was related to mariadb problems, today it’s NFS.

Not trying to bash Arch here, there is a very small team working on Arch and they overall do a good job when you take into account their team size. Also stuff is going to break occasionally when you’re always making stuff better.

On a side note, the streak counts seem to be working.

[Wayne Workman]

Between 1:40 AM and 2:44 AM CST, the Arch team likes to make changes. I can tell because working branch runs first, and succeeded with Arch, but later dev-branch and master failed with the same error.

[Wayne Workman]

FYI, lots of failures last night. On master branch, all Fedoras and Ubuntus failed, and then Ubuntu 17 failed on Dev branch - this is the only one that returned a log, it’s the ‘alter user’ mysql issue.

I am re-running the tests for the master branch to see if the same results happen or not.

[Wayne Workman]

Wanted to post a quick status of the testing’s streaks. You can see them in the link in my signature.

Currently, we have 4 Linux distributions that are neck & neck for the title of ultimate stability on the FOG Master branch.

We have a current streak of 40 for these Linux distributions:

• CentOS 7
• Debian 8
• Debian 9
• Ubuntu 16

We will see which Linux distribution pulls ahead in the long haul these coming months (and years). The race is ON!

[george1421]

@wayne-workman Well done on the stats. I think we have a good bunch of winners already.

[Wayne Workman]

Got a error on Arch across all branches this morning from apache:

[Mon Dec 04 07:43:13.301115 2017] [proxy_fcgi:error] [pid 2547:tid 140160016578304] [client 10.0.0.26:34968] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Error: Call to undefined function _() in /srv/http/fog/commons/text.php:22\nStack trace:\n#0 /srv/http/fog/commons/base.inc.php(41): require()\n#1 /srv/http/fog/management/index.php(22): require('/srv/http/fog/c...')\n#2 {main}\n  thrown in /srv/http/fog/commons/text.php on line 22\n'

Looks like something in Arch’s repo broke this since it was in all branches. Other distributions are fine.

[Sebastian Roth]

@Wayne-Workman Thanks for letting us know. Your tests distro tests are awesome man!

Arch has gone from PHP 7.1.12 to 7.2 lately - see here. The function call _(...) is an alias for gettext(...) which is used for translation strings. But I haven’t found why there should be an issue with that yet.

Can you do me a favor? On one of the Arch installations - if you still have one around - please create a file info.php in FOG web root directory with the following content:

<?php
phpinfo();
?>

Save and open in browser http://x.x.x.x/fog/info.php to see if you can figure if gettext extension is on or not. On my server I see “GetText Support enabled”

[Tom Elliott]

@sebastian-roth get text must be enabled in arch though I’m pretty sure we do this already.

I am aware 7.2 removes native mcrypt functions though which will impact things such as the client authentication. I haven’t had time to figure out a better alternative to replace these calls yet sorry.

[Wayne Workman]

@sebastian-roth Arch’s default apache root dir is very interesting: /srv/http
Anyways, I added that file to there and when viewing it via browser, I don’t see any indicators that gettext is on. Here’s a copy of the page:
https://mega.nz/#!EskiSSwT!Aj2px-ZMpwcDj90TBbpJRdB6G04AcBhrMR5WcPwlj6g

[Sebastian Roth]

@Wayne-Workman Thanks for that. Looking through it I found --with-gettext=shared in the compile options. So enabling the shared module still needs to be done. As Tom said this should be done by the installer. Ahhh, I think I know what’s up. The installer does regex-matching to enable the modules - see here:

...
sed -i 's/;extension=gettext.so/extension=gettext.so/g' $phpini ...
...

Now what changed in Arch is that they don’t use the .so extension in the configuration anymore. See here. I’ll fix our regex today.

As we don’t backport fixes to the master branch this one will be screwed with Arch for a long time I am afraid.

[Wayne Workman]

@sebastian-roth said in Fog Installer - Distro check:

As we don’t backport fixes to the master branch this one will be screwed with Arch for a long time I am afraid.

That’s how it should be. Also, looks like working branch failed again today.

[Sebastian Roth]

@Wayne-Workman Ok, I just pushed a fix (hope I got that right) to the working branch. Let’s see if that’s doing the job. If so I might push it to dev-branch soon as well.

[Wayne Workman]

@sebastian-roth Looks like you fixed it, good job man.

[george1421]

@wayne-workman Thank you for keeping focus on these distro checks!!

[Wayne Workman]

These distributions have completed 73 successful clean installations in a row using their latest patches with the FOG master branch:

• CentOS 7
• Debian 8
• Debian 9
• Ubuntu 16

We’re still waiting for a single winner to rise above the rest. The race for most-stable distribution continues.

[Wayne Workman]

@developers @moderators There were a ton of failures last night. Not sure why. When I get home this evening I’ll check it out.

[Wayne Workman]

@developers @moderators I’ve determined most of these failures are due to the remi repository being broken somehow. Some errors from CentOS 7:

package remi-release-7 is not installed
curl: (22) The requested URL returned error: 503 Service Temporarily Unavailable
error: skipping http://rpms.remirepo.net/enterprise/remi-release-7.rpm - transfer failed
Retrieving http://rpms.remirepo.net/enterprise/remi-release-7.rpm
curl: (22) The requested URL returned error: 503 Service Temporarily Unavailable
error: http://rpms.remirepo.net/RPM-GPG-KEY-remi: import read failed(2).
../lib/common/functions.sh: line 580: yum-config-manager: command not found
../lib/common/functions.sh: line 581: yum-config-manager: command not found

I checked on the Fedora ones, similar errors.

However, I’ve tried to reproduce the errors manually and I cannot.

I have to conclude that there was in fact a problem of some kind with the Remi repository, and that it now appears to be fixed. As far as Arch Linux goes, I really don’t have any idea. Everyone’s welcome to look at the logs though.

[Sebastian Roth]

@Wayne-Workman Thanks for looking into this. I maybe found what’s wrong on Arch. MariaDB says:

Installation of system tables failed! Examine the logs in /var/lib/mysql for more information.

DB seems to be not up. Possibly just a rename of the init file in Arch as I saw errors about that too.

[Wayne Workman]

@sebastian-roth Everything is back to the way it was this morning, with no effort from me.

So, we no longer have much of a tie going on. Debian 8, Debian 9, and Ubuntu 16 are now tied for FOG stability. CentOS didn’t fail out because of anything CentOS did, but rather because of the dependency on the remi repo to get certain PHP packages that FOG needs. I’m confident that if CentOS didn’t need the remi repo for FOG, it’d still be tied for leader. But it is what it is I guess.

It would be nice if we could in the future not require external repositories beyond what the OS ships with (like epel and remi and others).

[Sebastian Roth]

@Wayne-Workman Yes, good point. We should look into what is still needed. I think a lot of it came in when PHP7 was still new…