Fog Installer - Distro check

[Wayne Workman]

December 30th, 2016
Branch: master
Git Commit f23fb06ace4df6e9e69f974244baa032d1be4e04
Release Version: FOG 1.3.0

There is a minor hiccup in the FOG 1.3.0 installer that causes problems with detecting the router address on Arch, Fedora 24, Fedora 25, CentOS 7, Debian 8, and Ubuntu 16.04. While using the -y argument of the installer, most of these distributions will go into an infinite loop on a fresh installation. They will install without the -y argument but you will need to answer n for setting a router address or type in the actual router address when it asks. Ubuntu doesn’t seem to loop for me but still the router address is wrong in the settings.

There is a workaround for all of the distributions for using the -y argument. You would simply fill the routeraddress variable manually while you start the installer. My router address is 10.0.0.1 thus the answer is:

routeraddress=10.0.0.1 ./installfog.sh -y

You can use this method for a successful unattended installation.

Other than that, everything appears to be fine.

[Wayne Workman]

Janurary 31st, 2017
Branch: master
Git Commit: 7bde5a208dc9b681a22946c3788c5f8242da9f3a
Release Version: FOG 1.3.4

Debian 8.7 - fully updated - installed without issues.
Ubuntu 14.04 - fully updated - installed without issues
Ubuntu 16.04 - fully updated - installed without issues
CentOS 7 - fully updated - installed without issues per wiki instructions

Fedora 25 - fully updated - failed
@Tom-Elliott

* Setting up and starting RPCBind.............................Failed!

Below is the error log.
0_1485920319584_fog_error_1.3.4.log

This is what shows in journalctl -xe after I run systemctl restart rpcbind

-- Unit rpcbind.service has begun starting up.
Jan 31 21:39:21 localhost.localdomain rpcbind[10604]: rpcbind: /run/rpcbind/rpcbind.lock: No such file or directory
Jan 31 21:39:21 localhost.localdomain audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='
Jan 31 21:39:21 localhost.localdomain systemd[1]: rpcbind.service: Main process exited, code=exited, status=1/FAILURE
Jan 31 21:39:21 localhost.localdomain systemd[1]: Failed to start RPC Bind.
-- Subject: Unit rpcbind.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit rpcbind.service has failed.
-- 
-- The result is failed.
Jan 31 21:39:21 localhost.localdomain systemd[1]: rpcbind.service: Unit entered failed state.
Jan 31 21:39:21 localhost.localdomain systemd[1]: rpcbind.service: Failed with result 'exit-code'.

Also have tried uninstalling rpcbind, the service still will not restart.

[Wayne Workman]

Just so people are aware, I’m still testing the distributions. I’ve automated this & the results are available daily to the fog team only. I’ll post here with concerns.

Right now - Fedora 25 is broke, it’s a bug in the OS, nothing we can do about it. It’s related to rpcbind.

And it appears that starting last night something is wrong with the fog installation on Ubuntu 16. It’s related to the alter user command in MySQL.

[Wayne Workman]

After re-running the tests, Ubuntu 16 appears fine. Must have been a fluke. Anyone want to donate a server? I’m using an old Poweredge r410.

[Wayne Workman]

Fedora 25 and 26 are not working with the installer right now, something to do with rpcbind.

I’ve not found the time to look at it. The failure logs are here if anyone want’s to take a peek:
http://perpetuum.io:20080/fog_distro_check/Fedora25/fog/2017-07-31_04-14_fog.log
http://perpetuum.io:20080/fog_distro_check/Fedora26/fog/2017-07-31_04-22_fog.log

Of course, I’m still testing the Linux distributions that FOG tries to support, it’s all automated now. You can find daily results in the dashboard link in my signature.

[Sebastian Roth]

Seems like they had several issues with rpcbind over the last year or so: https://bugzilla.redhat.com/show_bug.cgi?id=1401561 and https://bugzilla.redhat.com/show_bug.cgi?id=1415496 (Not sure if those are related or not!)

@Wayne-Workman SElinux is disabled on those machines I suspect?!

[Tom Elliott]

@sebastian-roth I suspect it was my “fixing” code that is now “fixed” but causing a problem with latest updates. Just a guess. My fixing code just made it so fedora could install, and basically all it did was make sure the path it was looking for during startup existed. Now it looks like it’s the reverse, it’s trying to create a file that already exists, hence permission denied… (as it’s likely in use then too).

[Wayne Workman]

@sebastian-roth said in Fog Installer - Distro check:

SElinux is disabled on those machines I suspect?!

I forgot to disable it on those two VMs… that is my bad. I’ll get that fixed.

[Wayne Workman]

The issues appear to have been non-issues in the latest tests. SELinux was not set to permissive (my fault).

[Wayne Workman]

Soon, the fog installer dashboard (see my signature) will have streak counts for each item. There will be a current success streak and a record success streak.

I’ve been wanting to do this for a while, I think it’ll be fun to watch the counts stack up. I’m monitoring the files for a couple days to be sure everything is working right before I put the values into the tables.

[Wayne Workman]

Streak counts are now public.

[george1421]

Very nice job!! After some time we should be able to identify the most stable OS for FOG installs. This will really come in handy when thinking about which OS we might support in the future.

[Wayne Workman]

@george1421 I think I already know which ones will be most stable. I’ll keep quiet though and we’ll see how it goes.

[Wayne Workman]

Arch bombed out this morning on all branches - something to do with NFS. This is typical for Arch - one day it’ll just bomb out hard across the board on the same item, a couple days later it’s fixed. Once it was related to mariadb problems, today it’s NFS.

Not trying to bash Arch here, there is a very small team working on Arch and they overall do a good job when you take into account their team size. Also stuff is going to break occasionally when you’re always making stuff better.

On a side note, the streak counts seem to be working.

[Wayne Workman]

Between 1:40 AM and 2:44 AM CST, the Arch team likes to make changes. I can tell because working branch runs first, and succeeded with Arch, but later dev-branch and master failed with the same error.

[Wayne Workman]

FYI, lots of failures last night. On master branch, all Fedoras and Ubuntus failed, and then Ubuntu 17 failed on Dev branch - this is the only one that returned a log, it’s the ‘alter user’ mysql issue.

I am re-running the tests for the master branch to see if the same results happen or not.

[Wayne Workman]

Wanted to post a quick status of the testing’s streaks. You can see them in the link in my signature.

Currently, we have 4 Linux distributions that are neck & neck for the title of ultimate stability on the FOG Master branch.

We have a current streak of 40 for these Linux distributions:

• CentOS 7
• Debian 8
• Debian 9
• Ubuntu 16

We will see which Linux distribution pulls ahead in the long haul these coming months (and years). The race is ON!

[george1421]

@wayne-workman Well done on the stats. I think we have a good bunch of winners already.

[Wayne Workman]

Got a error on Arch across all branches this morning from apache:

[Mon Dec 04 07:43:13.301115 2017] [proxy_fcgi:error] [pid 2547:tid 140160016578304] [client 10.0.0.26:34968] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Error: Call to undefined function _() in /srv/http/fog/commons/text.php:22\nStack trace:\n#0 /srv/http/fog/commons/base.inc.php(41): require()\n#1 /srv/http/fog/management/index.php(22): require('/srv/http/fog/c...')\n#2 {main}\n  thrown in /srv/http/fog/commons/text.php on line 22\n'

Looks like something in Arch’s repo broke this since it was in all branches. Other distributions are fine.

[Sebastian Roth]

@Wayne-Workman Thanks for letting us know. Your tests distro tests are awesome man!

Arch has gone from PHP 7.1.12 to 7.2 lately - see here. The function call _(...) is an alias for gettext(...) which is used for translation strings. But I haven’t found why there should be an issue with that yet.

Can you do me a favor? On one of the Arch installations - if you still have one around - please create a file info.php in FOG web root directory with the following content:

<?php
phpinfo();
?>

Save and open in browser http://x.x.x.x/fog/info.php to see if you can figure if gettext extension is on or not. On my server I see “GetText Support enabled”