Improve documentation

[A Former User]

@Sebastian-Roth if you think a separate repo is better, than that’s fine for me and it is good to keep it under the fogproject. We can place release tags in that project and achieve the same goal.

@Wayne-Workman yes let’s start small. From my experience with setting up Fog project as a sysadmin/network engineer I am interested in the following parts:

• How does Fog work? A small introduction and a description of components and how it fits in the network. The last part is important for network admins to understand and support a fog installation in their network
• Installation instructions on Debian and Redhat of a fog server and fog storage server
• Managing fog (the Web UI)
• Doing your first image capture/deployment
• Reference of all configuration options

Later we will have to discuss whether to include advanced options like exotic installs, FAQ, Troubleshooting although I think that the forums already play a great role in this and maybe also the wiki.

If you agree, then let me assemble the first pages. The wiki provides lots of information that is very usable, so that’s just copy/paste and reformat.

[Sebastian Roth]

@Jurgen-Goedbloed said in Improve documentation:

If you agree, then let me assemble the first pages. The wiki provides lots of information that is very usable, so that’s just copy/paste and reformat.

Sounds great to me!

From a practical point of view I will organize for you to have write/push access to the fog-docs repo - and everyone else who is keen to work on this (please send me a PM). Sure there is another way by forking that repo to your own account and sending in pull requests but that adds extra work for you and us and I don’t think that’s apropriate.

[JJ Fullmer]

@Sebastian-Roth and @Jurgen-Goedbloed Could I also be added as a maintainer. I would love to help with this project.
Maybe we make some github issues for what documents need creating and or copy pasting and divide and conquer?

[JJ Fullmer]

@Jurgen-Goedbloed
I would actually strongly suggest we use mkdocs instead of sphinx.
mkdocs works better with markdown from what I’ve read and experienced and it’s a lot easier to setup and learn.
Considering the forum uses markdown, the existing wiki uses markdown, I think it would be a better option for the project.
Markdown is also a lot easier to learn and to write and I feel like it would make it a lot easier for future maintainers to pick up and run with.

[Sebastian Roth]

@JJ-Fullmer While Markdown might be easier in the first place there are a few reasons not to use it: https://www.ericholscher.com/blog/2016/mar/15/dont-use-markdown-for-technical-docs/ (linked from readthedocs.io)

[A Former User]

@JJ-Fullmer I have set up the first documentation in sphinx now. I agree with you that it is more difficult than markdown. I have no development experience and needed to learn sphinx from scratch, but having read Sebastian’s comment, let’s start with sphinx. I think when the ‘skeleton’ is set up, it will be much easier to contribute as you have examples.

[JJ Fullmer]

@Sebastian-Roth I just read through that and I see his points but it’s from 4 years ago. From my experience with markdown commonmark is now widely used. Both sphinx and mkdocs use commonmark.
However, considering it sounds like we want the easily navigable versions, sphinx may be the way to go as mkdocs in readthedocs only has one theme that shows the readthedocs version dropdown (I messed with a bunch of themes yesterday). Granted sphinx can be configured to use markdown, but I suppose I can learn another documentation language.

[JJ Fullmer]

@Jurgen-Goedbloed
Are you able to see the built readthedocs page?
I’m having trouble getting it to build locally to see what it’s going to look like and when I go to docs.fogproject.org I get a 403 error. I think that @Joe-Schmitt may have set some security up on that page on the fogproject domain while it was to be under construction. Maybe we should host it on the readthedocs domain until we get that figured out? That’s a setting in the readthedocs project that I can’t access.

edit:
Figured out my build problems, so can at least browse it locally. Updated the readme with build instructions and added a make.ps1 for easier building.

[A Former User]

@JJ-Fullmer No, I’m not. I see the same as you, an nginx 403 page.
I think that for the domain fogproject.org a DNS CNAME record should be made:
docs.fogproject.org IN CNAME readthedocs.io

I have admin access to readthedocs and I have removed this domain temporary until the DNS is fixed. For now you can see the first results under https://fogproject.readthedocs.io

The initial structure is now in github and I think it is best if we spread the documentation across many files, as this decreases the chance of having merge conflicts when together working on the documentation.

[Sebastian Roth]

@Jurgen-Goedbloed said in Improve documentation:

I think that for the domain fogproject.org a DNS CNAME record should be made:
docs.fogproject.org IN CNAME readthedocs.io

We have NGINX running on our webserver. I just had a quick look at the config and for https://docs.fogproject.org we have a proxy_pass (reverse proxy) setting to https://fogproject.readthedocs.io:443

When loading the page I get this in the error logs:

2020/10/07 15:02:08 [error] 26062#26062: *2758 connect() to [2606:4700::6811:2052]:443 failed (101: Network is unreachable) while connecting to upstream, client: 91.18.10.171, server: docs.fogproject.org, request: "GET / HTTP/2.0", upstream: "https://[2606:4700::6811:2052]:443/", host: "docs.fogproject.org"

Interesting it resolves to an IPv6 address but is unable to connect to that. I will take a look at this tomorrow morning.

[JJ Fullmer]

@Sebastian-Roth @Jurgen-Goedbloed
I believe that fogproject uses cloudflare. There’s a note about using cloudflare dns in the readthedocs docs.
https://docs.readthedocs.io/en/stable/custom_domains.html?highlight=domains#notes-for-cloudflare-dns-users

Doesn’t look like we’re getting that same error as described in their documentation but may still be related.

I think that reverseproxy might be what Joe set up 3 years ago but it looks like that feature is depreciated. https://docs.readthedocs.io/en/stable/custom_domains.html?highlight=domains#proxy-ssl

[Sebastian Roth]

@JJ-Fullmer Thanks for the hint! Seems like reverse proxy setup was used earlier but now readthedocs is able to provide SSL certificates for external domains (interesting concept I find) through cloudflare. We’ll setup a CNAME soon and und the meantime we’ll use https://fogproject.readthedocs.io/

[JJ Fullmer]

@Sebastian-Roth @Wayne-Workman @Jurgen-Goedbloed
So I’m going through the management section of the wiki, trying to do one page/section a day converting them to the new one. Going back through and updating them will be a separate thing I think.
There are some old flash video tutorials hosted in places like http://freeghost.sourceforge.net/videotutorials/hostinfo.html

Do we want to recreate these in a newer not about to be depreciated format? And with the new gui?
We could also just host them in github, especially for ones that don’t require sound and can just be .gif files.

For now I can just embed the old links, but especially these flash ones probably need to be updated or removed, we should just have a standard as we migrate I think. There’s also the factor that the fog gui will change again in 1.6.

[Wayne Workman]

@JJ-Fullmer can they be converted to something YouTube supports? (as opposed to re-recording)

We can put them on the FOG Project Videos YouTube channel. https://www.youtube.com/channel/UCrvOQPcm1SDIfIrzWZ9K3bA/videos

[Sebastian Roth]

@JJ-Fullmer Thanks you are pushing this forward way quicker than I would have imagined.

especially for ones that don’t require sound and can just be .gif files.

While I really like the idea of keeping things as simple as possible I wonder if GIF is helpful with content that you might want to pause or skip through.

There’s also the factor that the fog gui will change again in 1.6.

That’s a good point. We might want to wait with updating the video stuff until 1.6 is on the doorstep. Just my 50 cents.

[JJ Fullmer]

@Wayne-Workman said in Improve documentation:

@JJ-Fullmer can they be converted to something YouTube supports? (as opposed to re-recording)

We can put them on the FOG Project Videos YouTube channel. https://www.youtube.com/channel/UCrvOQPcm1SDIfIrzWZ9K3bA/videos

I think this would be an excellent method, gives us extra search visibility too.

@Sebastian-Roth Good point on the gif. I was just thinking of what I’ve seen in other documentation recently. It’s cool for showing how quick and easy some things are but pause buttons are nice.

Read-the-docs has a youtube video embedded in their docs so I took that as an example and did a test with a random youtube video from our channel
https://fogproject.readthedocs.io/en/latest/management/index.html#hosts

Looks good to me.

I am having trouble getting the swf converted though. When I run it through handbrake or any online convert tool it comes out as a jumbled mess. It’s also a video showing the 0.12 fog gui. Maybe it’s not worth it, I very much doubt anyone uses that version anymore.

[JJ Fullmer]

Well I found one video converter that worked. But it was just the trial of it so I only got half of the video. They want $45 though.
https://www.faasoft.com/video-converter.html

Not sure if that’s worth it for the flash videos of the pre 1.0 fog gui.

[Sebastian Roth]

@JJ-Fullmer said in Improve documentation:

Not sure if that’s worth it for the flash videos of the pre 1.0 fog gui.

I don’t think it’s worth it. We need to create a new one anyway.

[Wayne Workman]

agree with Sebastian. Just link to the old video. When it dies, it dies.

[A Former User]

I am now working on the installation part of the manual.

Can we agree on the starting point that, when coping things from the wiki to the new documentation that we will only focus on the current (1.5.9) version? So all parts that refer to old versions will not be copied and that we will also use screenshots of the current version. I think this will help a lot in understanding and recognition for new Fog users.