As I sit here working on making the daily fog installation tests more reliable, I wondered to myself when I should remove the older OSs (centos7, rhel7, etc). I thought it’d be nice to know how often what OS is being used to install FOG on. So I propose the following.
I can create a publicly available API that simply accepts a “string” being submitted. Using this, I can put together a popularity contest type thing that will allow us to see what people are installing FOG on. I can build the API (I have scaffolding that I can use to get this done pretty fast) and run it, and make the results publicly viewable.
Within the FOG Installer scripts, I can add a question about participating in the popularity contest. I think this should default to “Yes”. If “Yes” is chosen, a simple curl call is executed sending the output of
lsb_release --allto the API.
Thoughts on this? I’m willing to put it all together and submit the pull requests for it. Though wanted feedback before I went after it.
I suggest we look at both locations and send the one from the file with a newer timestamp.
While that would surely work I would really like to untangle this convoluted solution now and for ever. The more I think about it the less I want to add more if then else, check this, link here, try again, whatever. Let’s discuss the details in a new topic.
Please leave the reporting script as it is right now. It’s really good you added it this way so we got notice of this.
good point… I suggest we look at both locations and send the one from the file with a newer timestamp.
Or maybe there is a better way?
Looking at the data I am wondering how versions 1.5.2, 1.5.5 and 1.5.6 can make it into the database. From my point of view this is a good sign those installs still have an old version of the web UI sitting in
/var/www/fogand the new one in
While I would hope we can figure out a way to detect those situations and let people know I would still think that we need a more generic way of grabbing the version info instead of using a static path for system.class.php.
@Wayne-Workman Sounds awesome but right now I get a “Bad Gateway” message instead.
Here is my first pass at the presentation layer: http://fog-external-reporting-results.theworkmans.us
The graphs/files/stats automatically are updated every hour. I think the graphs could use color adjustments, but overall I’m pretty satisfied. What do you all think? BTW, the data in here is real!
The next phase of this is abuse detection for the data reported. I need to scan for profanity, insults, etc; to keep the graphs clean. Python has tools for all of that.
How exciting, entries! A whole lot of Ubuntu, one of them is Ubuntu 16!
FOG External Reporting sounds good to me. I’ll get the endpoint name changed sometime tonight and submit a PR for that.
I’m still needing to put together the last pieces of this, the presentation layer. This will be coming over the next couple weeks. Plans are to render some basic bar graphs showing the information gathered from the last 7 days, and to make available the database dump. Probably a cron-job to refresh those things every hour.
Instead of Analytics, or Rainman (though I love this movie), or “Install survey”
Something that’s more explicit to what it’s doing:
FOG External Reporting
@Junkhacker working-1.6 has been updated with this.
I agree that most will not mind.
I just didn’t want people to get into this unknowningly.
I don’t know how many people use the -y switch, so that was why I wanted to “err” on keeping things as they are.
@Tom-Elliott i think we should default to Y/n instead of y/N
most people don’t care one way or the other and will just click through, let the people who care chose not to send it
Added ability to turn on/off this “feature”.
FOG would like to collect some data: We would like to collect the following information: 1. OS Name (CentOS, RedHat, Debian, etc....) 2. OS Version (8.0.2004, 7.2.1409, 9, etc....) 3. FOG Version (1.5.9, 1.6, etc....) What is this information used for? We would like to simply track the common types of OS being used, along with the OS Version, and the various versions of FOG being used. Are you ok with sending this information? [Y/n]
When Selected either yes or no, the screen will tell status as well:
* Here are the settings FOG will use: * Base Linux: Redhat * Detected Linux Distribution: CentOS Linux * Server IP Address: .... * Server Subnet Mask: .... * Hostname: fogserver * Installation Type: Normal Server * Internationalization: 0 * Image Storage Location: /images * Using FOG DHCP: Yes * DHCP router Address: ... * Send OS Name, OS Version, and FOG Version: Yes
* Here are the settings FOG will use: * Base Linux: Redhat * Detected Linux Distribution: CentOS Linux * Server IP Address: .... * Server Subnet Mask: .... * Hostname: fogserver * Installation Type: Normal Server * Internationalization: 0 * Image Storage Location: /images * Using FOG DHCP: Yes * DHCP router Address: ... * Send OS Name, OS Version, and FOG Version: No
information resulting from the systematic analysis of data or statistics.
What are we gathering,
OS Name/Type, OS Version, FOG Version
What are we doing?
Analyzing how many of a type of OS and version, along with the number of “versions” run out in the wild.
While the data we’re gathering is, right now, only very basic, I think analytics is still the true term.
If we decide analytics is not the term to use, I still think it’s important to add (and I’m working on doing that) a way for the admins to disable if they don’t want their information being reported. We’re not doing much or collecting any metrics of the data in the sense of IPs or how we can best improve our code base. We are, after all, collecting and sending data to another source.
This information we are gathering is still a form of data we can use to gather what Operating Systems and versions, along with the general FOG version being used in the wild.
@Wayne-Workman how about “install survey”
Maybe we call it “Rainman”?
I think analytics is the appropriate term, even if the information we’re using is minimal.
I don’t agree! The term “analytics” implies way more than we do and it might turn down people who use FOG and don’t like analytic stuff that is going on all over the web.
Why should we name it something we don’t want to do - and I am absolutely sure I never add real analytics stuff to FOG myself.
I’ve validated the installation code as I’ve written it does work against all the major distributions - and I’ve also discovered my daily test for Ubuntu 20 was really Ubuntu 18 (fixed now).
Below is what the data looks like in my test environment. It’s interesting to note that RedHat changes it’s distribution name from major release to major release. It’s of no consequence for our use, but I think that’s interesting.
The server-side’s code is in this PR:
The FOG Server’s code is in this PR:
@Wayne-Workman I think that’d work best, yes.
@Tom-Elliott Looks fine to me. Should I submit a pull request to the
dev-branchsince you’ve already added this to the 1.6 branch?