Popularity Contest

[Tom Elliott]

Added ability to turn on/off this “feature”.

  FOG would like to collect some data:
      We would like to collect the following information:
        1. OS Name (CentOS, RedHat, Debian, etc....)
        2. OS Version (8.0.2004, 7.2.1409, 9, etc....)
        3. FOG Version (1.5.9, 1.6, etc....)

  What is this information used for?
      We would like to simply track the common types of OS
      being used, along with the OS Version, and the various
      versions of FOG being used.

  Are you ok with sending this information? [Y/n]

When Selected either yes or no, the screen will tell status as well:

 * Here are the settings FOG will use:
 * Base Linux: Redhat
 * Detected Linux Distribution: CentOS Linux
 * Server IP Address: ....
 * Server Subnet Mask: ....
 * Hostname: fogserver
 * Installation Type: Normal Server
 * Internationalization: 0
 * Image Storage Location: /images
 * Using FOG DHCP: Yes
 * DHCP router Address: ...
 * Send OS Name, OS Version, and FOG Version: Yes

Or:

 * Here are the settings FOG will use:
 * Base Linux: Redhat
 * Detected Linux Distribution: CentOS Linux
 * Server IP Address: ....
 * Server Subnet Mask: ....
 * Hostname: fogserver
 * Installation Type: Normal Server
 * Internationalization: 0
 * Image Storage Location: /images
 * Using FOG DHCP: Yes
 * DHCP router Address: ...
 * Send OS Name, OS Version, and FOG Version: No

[Junkhacker]

@Tom-Elliott i think we should default to Y/n instead of y/N

most people don’t care one way or the other and will just click through, let the people who care chose not to send it

[Tom Elliott]

@Junkhacker working-1.6 has been updated with this.

I agree that most will not mind.

I just didn’t want people to get into this unknowningly.

I don’t know how many people use the -y switch, so that was why I wanted to “err” on keeping things as they are.

[Tom Elliott]

Instead of Analytics, or Rainman (though I love this movie), or “Install survey”

Something that’s more explicit to what it’s doing:

FOG External Reporting

[Wayne Workman]

FOG External Reporting sounds good to me. I’ll get the endpoint name changed sometime tonight and submit a PR for that.

I’m still needing to put together the last pieces of this, the presentation layer. This will be coming over the next couple weeks. Plans are to render some basic bar graphs showing the information gathered from the last 7 days, and to make available the database dump. Probably a cron-job to refresh those things every hour.

[Wayne Workman]

How exciting, entries! A whole lot of Ubuntu, one of them is Ubuntu 16!

[Wayne Workman]

Here is my first pass at the presentation layer: http://fog-external-reporting-results.theworkmans.us

The graphs/files/stats automatically are updated every hour. I think the graphs could use color adjustments, but overall I’m pretty satisfied. What do you all think? BTW, the data in here is real!

The next phase of this is abuse detection for the data reported. I need to scan for profanity, insults, etc; to keep the graphs clean. Python has tools for all of that.

[Sebastian Roth]

@Wayne-Workman Sounds awesome but right now I get a “Bad Gateway” message instead.

[Wayne Workman]

Fixed: http://fog-external-reporting-results.theworkmans.us/

[Sebastian Roth]

@Wayne-Workman Nice!

Looking at the data I am wondering how versions 1.5.2, 1.5.5 and 1.5.6 can make it into the database. From my point of view this is a good sign those installs still have an old version of the web UI sitting in /var/www/fog and the new one in /var/www/html/fog.

While I would hope we can figure out a way to detect those situations and let people know I would still think that we need a more generic way of grabbing the version info instead of using a static path for system.class.php.

[Wayne Workman]

good point… I suggest we look at both locations and send the one from the file with a newer timestamp.

Or maybe there is a better way?

[Sebastian Roth]

@Wayne-Workman said in Popularity Contest:

I suggest we look at both locations and send the one from the file with a newer timestamp.

While that would surely work I would really like to untangle this convoluted solution now and for ever. The more I think about it the less I want to add more if then else, check this, link here, try again, whatever. Let’s discuss the details in a new topic.

Please leave the reporting script as it is right now. It’s really good you added it this way so we got notice of this.

[Wayne Workman]

Thoughts on the external reporting DNS name?

Right now it’s: https://fog-external-reporting-entries.theworkmans.us:/api/records

This works fine, but it’s not a fogproject.org address.

If nobody has an opinion, I’m happy to leave it as-is forever, but I think it may be more appropriate if there’s an A Record created like fog-external-reporting-entries.fogproject.org. This can be set to the public IP of the server I’ve setup to record the entries, and also could be easily changed in the future if needed. I’m not immortal, nor is anyone else here so this is why I bring it up.

We really need to get it the way we want it before the next release.

[Sebastian Roth]

@george1421 Good point! Though I would use a more simple name that is easy to remember and type, e.g. stats.fogproject.org?

[Wayne Workman]

Sounds good.

[Sebastian Roth]

@Wayne-Workman said in Popularity Contest:

Right now it’s: https://fog-external-reporting-entries.theworkmans.us:/api/records

This URL does not seem to work in the browser. It’s the backend API URL if I am not on the wrong track. But we want the frontend URL http://fog-external-reporting-results.theworkmans.us/ right? Can you setup HTTPS as well?

[Wayne Workman]

Yeah the back-end only takes entries, nothing else.

The frontend url is less important, as that’s not needed in the fog codebase anywhere. It’s just a place one visits to see results. I can get HTTPS setup for it.

[Wayne Workman]

@Sebastian-Roth So when I try:

curl -X POST -H "Content-Type: application/json"  -d '{"fog_version":"testing","os_name":"testing","os_version":"testing"}' https://stats-api.fogproject.org/api/records

This is the result:

curl: (6) Could not resolve host: stats-api.fogproject.org

[Wayne Workman]

@Sebastian-Roth if you can get an A record created for stats-api.fogproject.org with a value of 3.20.18.241 I can get the ssl cert and such fixed on my side (using lets encrypt). Once that’s all done, we can update the reporting script to use the new URL.

For the results to have a good redirect from stats.fogproject.org, I need the SSL cert to include that subject alternative name. Please create this record, it’ll allow me to create that SSL cert. These values below are not secrets, and they will timeout in 72 hours from now (sometime wednesday). if it’s not made by then I can get new values. This is part of how AWS verifies the cert your making is legit.

Name:
_395058431eb78e5f8f9cc2478adbdfa3.stats.fogproject.org.

Type:
CNAME

Value:
_ee8469a4f9a938de348ef383be83b11f.zbkrxsrfvj.acm-validations.aws.

If this can’t be done for whatever reason, I can use the original results URL and it’ll work fine, just redirects from stats.fogproject.org won’t work.

Here’s the changes on the cloud side if you were interested.

[Sebastian Roth]

@wayne-workman said in Popularity Contest:

For the results to have a good redirect from stats.fogproject.org, I need the SSL cert to include that subject alternative name.

That’s interesting. While I can look it up using the host command I can’t ping it.

$ host stats.fogproject.org
stats.fogproject.org is an alias for fog-external-reporting-results.theworkmans.us.
$ ping stats.fogproject.org
ping: stats.fogproject.org: Name or service not known