Let's make scripts!

[Wayne Workman]

I would like to suggest making troubleshooting scripts for every major flavor of Linux, for FOG troubleshooting.

The scripts should test every relevant service, maybe run basic diagnostics on those services… The scripts could even pull data from configuration files, and definitely the FOG install log.

All the scripts should put all output into a single text file, for FOG users to post here.

This should expedite trouble shooting, because we can say, Here, run this script, give us the output (or they themselves use it as a troubleshooting tool).

We should make a WiKi section just for these.

---

EDIT

This project has been converted into a SourceForge project.

Here is how you can get it:

svn checkout svn://svn.code.sf.net/p/fog-troubleshoot-script/svn/ /svn/fog-troubleshoot-script

Here’s a link to the project: https://sourceforge.net/projects/fog-troubleshoot-script/

Many many thanks to everyone that was involved with improving this script. Hopefully people will continue to collaborate and help to improve this FOG side project. If you would like commit access, just ask me or @Tom-Elliott

[Wayne Workman]

This post is deleted!

[Sebastian Roth]

I quite like the idea but I am wondering if it’s worth the effort to create those scripts?! Most problems are very special and some involve switch configurations, DHCP settings on external (windows) servers and possibly other clients are needed to check boot menu via HTTP or NFS mounts.
But feel free to start scripting and we’ll jump in as things get rolling. What do you think?!

[Wayne Workman]

[quote=“Uncle Frank, post: 43426, member: 28116”]I quite like the idea but I am wondering if it’s worth the effort to create those scripts?! Most problems are very special and some involve switch configurations, DHCP settings on external (windows) servers and possibly other clients are needed to check boot menu via HTTP or NFS mounts.
But feel free to start scripting and we’ll jump in as things get rolling. What do you think?![/quote]

It’s a good opportunity to learn something about Linux scripting.

The first one will be on Fedora 21.

I’m going to rely heavily on the forum to figure out all the commands that need ran.

[Wayne Workman]

This is my first bash script ever, was really fun.

The idea behind having scripts like this is for language barriers and people new to Linux.

And, just a general starting point for troubleshooting.

[CODE]
echo ‘----------------------Fedora version below’
cat /etc/issue >> /tmp/Fedora21Fog.txt
echo ‘----------------------Installation log below’
cat /var/log/foginstall.log >> /tmp/Fedora21Fog.txt
echo ‘----------------------Apache Error log below’
cat /var/log/httpd/error_log >> /tmp/Fedora21Fog.txt
echo ‘----------------------TFTP Status below’
systemctl status xinetd.service >> /tmp/Fedora21Fog.txt
echo ‘----------------------RPC Status below’
systemctl status rpcbind >> /tmp/Fedora21Fog.txt
echo ‘----------------------NFS Status below’
systemctl status nfs-server >> /tmp/Fedora21Fog.txt
echo ‘----------------------FTP Status below’
systemctl status vsftpd.service >> /tmp/Fedora21Fog.txt
echo ‘----------------------Firewall Status below’
systemctl status firewalld.service >> /tmp/Fedora21Fog.txt
echo ‘----------------------SELinux Status below’
sestatus >> /tmp/Fedora21Fog.txt
echo ‘----------------------FOGMulticastManager Status below’
systemctl status FOGMulticastManager.service >> /tmp/Fedora21Fog.txt
echo ‘----------------------FOGImageReplicator Status below’
systemctl status FOGImageReplicator.service >> /tmp/Fedora21Fog.txt
echo ‘----------------------FOGSnapinReplicator Status below’
systemctl status FOGSnapinReplicator.service >> /tmp/Fedora21Fog.txt
echo ‘----------------------FOGScheduler Status below’
systemctl status FOGScheduler.service >> /tmp/Fedora21Fog.txt
echo ‘----------------------/etc/dnsmasq.d/ltsp.conf file below’
cat /etc/dnsmasq.d/ltsp.conf >> /tmp/Fedora21Fog.txt
echo ‘----------------------/etc/vsftpd/vsftpd.conf file below’
cat /etc/vsftpd/vsftpd.conf >> /tmp/Fedora21Fog.txt
echo ‘----------------------/etc/rc.d/rc.local file below’
cat /etc/rc.d/rc.local >> /tmp/Fedora21Fog.txt
echo ‘----------------------Check for /images/.mntcheck’
ls /images -a |grep “mntcheck” >> /tmp/Fedora21Fog.txt
echo ‘----------------------Check for /images/dev/.mntcheck’
ls /images/dev -a |grep “mntcheck” >> /tmp/Fedora21Fog.txt
echo ‘----------------------Contents of /opt/fog_trunk/packages/tftp’
ls /opt/fog_trunk/packages/tftp >> /tmp/Fedora21Fog.txt
[/CODE]

[Joe Schmitt]

I’m just going to throw my 2 cents in here. I think what you guys are doing is fantastic. It may be best to outline what debugging information is vital for us, and build the scripts off that. What you’ve posted is great, but I’d like to see a ‘standard’ for the scripts, a list of what information and in what format. That way you/we can build scripts for all platforms that output the same information in the same format.

If these scripts gain some ground, we could even include them in the repository.

[Wayne Workman]

I agree wholesomely.

So,

if the commands differ for a different distribution, we can just include all the commands, right?

The ones that don’t work will just error out.

I’ll see what I can throw together.

[Joseph Hales]

I second that emotion Most of what I do at this point is google and cut and paste to my command line while I am learning a lot as I go along it is a very steep learning curve and this would go a long way to helping you guys who have been so very helpful better understand what the hell we did to our servers this time.

[Wayne Workman]

Following up on this, I THINK I have the Fedora 21 portion completed…
You will find the file attached, along with the output as well (because it’s really long).

Ideally, the command groups will go to their designated OS output file.
For example, I’ve got Fedora 21 command output going to /tmp/Fedora21Fog.txt
Ubuntu could be /tmp/UbuntuX.xFog.txt or something.

If at all possible, I’d really like some help to convert all of these commands to Ubuntu equivalents, and eventually other OSs too.

Opinions/thoughts/contributions welcome and wanted.

[url=“/_imported_xf_attachments/1/1874_Script.zip?:”]Script.zip[/url]

[cml]

Hacked one together for Debian which may cover Ubuntu as well. I am not sure on the DHCP status and config portion, but I will set up a server to test tonight.

[url=“/_imported_xf_attachments/1/1887_DebianScript.zip?:”]DebianScript.zip[/url]

[ch3i]

Hi,

You can create a single script for each version using uname like this :
[CODE]DISTRIB_NAME=($(uname -v | grep -w -o “Debian|Ubuntu|Redhat|Fedora”)) # add other redhat/debian based

if [ “$DISTRIB_NAME” == “Debian” ] || [ “$DISTRIB_NAME” == “Ubuntu” ] || [ “$DISTRIB_NAME” == “Other_Debian_Based” ]
then
your instructions here;
elif [ “$DISTRIB_NAME” == “Fedora” ] || [ “$DISTRIB_NAME” == “Redhat” ] || [ “$DISTRIB_NAME” == “Other_Redhat_Based” ]
then
your instructions here;
fi[/CODE]

Regards,
Ch3i.

[cml]

Pulled code from the FOG installer scripts to come up with the combination. I left Arch commented out at the end of the script if anyone wants to give a go at it. If not I can after testing completes next week.

[url=“/_imported_xf_attachments/1/1891_FogScript.zip?:”]FogScript.zip[/url]

[Wayne Workman]

[quote=“cml, post: 45691, member: 28688”]cml[/quote]
Thank you for the logic, it’s exactly what was needed!

[quote=“ch3i, post: 45671, member: 2513”]Ch3i.[/quote]
Thanks for taking the time to convert the commands to Debian, and putting together all the code.

I’ve made a few small changes - mostly to simplify things a bit more.

I’ve changed the output path to whatever directory it’s run from.
I also changed the Fedora output from “Fedora21Fog.txt” to just “FedoraFog.txt”
I changed the name of the file to simply “troubleshoot.sh” for simplicity and obviousness.

It works beautifully on Fedora 21.

If anyone can think of anything else that should be included in the checks, suggestions are welcome.
If someone can test the Debian portions to see if they work on Ubuntu, that’d be great because Ubuntu is the most used distro for FOG.
And, as we can, get people to continue adding to the script for other distros of Linux.

Before you know it, we will have a one-stop-shop for information collection to aid in troubleshooting - be it here on the forums, or to help people with a starting place so they can help themselves.

The latest script is attached.

[B]OSs covered so far:[/B]
Fedora 20/21
Debian

[url=“/_imported_xf_attachments/1/1902_troubleshoot.zip?:”]troubleshoot.zip[/url]

[cml]

Few changes.

Ubuntu had to be run with sudo, so added a root check to the beginning of the file.
Ubuntu also has UFW as a firewall, added a check for that.
Changed the Debian/Ubuntu version check to ‘lsb_release -a’ to get more information.
Pointed the log file to a variable that is $Distro-Fog.txt
The script now overwrites the log if ran again.

Script and Debian/Ubuntu logs attached.

[url=“/_imported_xf_attachments/1/1907_troubleshoot.zip?:”]troubleshoot.zip[/url]

[Wayne Workman]

Tom helped with a command to convert the output file to windows-friendly text. I’ve added this at the very bottom of the script.

Latest and greatest is attached.

[url=“/_imported_xf_attachments/1/1910_troubleshoot.zip?:”]troubleshoot.zip[/url]

[Tom Elliott]

Here’s a more refined version. This particular should work on CentOS/Redhat (in general e.g. 5,6,7) and Fedora.

It also prints that the file is complete and where it can be located.

[url=“/_imported_xf_attachments/1/1911_troubleshoot.zip?:”]troubleshoot.zip[/url]

[Tom Elliott]

Here’s slightly more refined. Log items are somewhat nice now (more informative). Theres also a lot more checking.

troubleshoot.zip

[Wayne Workman]

Now that’s awesome. This is going to be a major help for troubleshooting!

[Joseph Hales]

Can we sticky this and add to the wikki.

[Joseph Hales]

When I ran the latest version in this thread I got the following output in terminal.
./troubleshoot.sh: line 19: [: : integer expression expected
rpcbind: unrecognized service
nfs: unrecognized service
iptables: unrecognized service
Usage FOGMulticastManager {start|stop|restart}
Usage FOGSnapinReplicator {start|stop|restart}
Usage FOGImageReplicator {start|stop|restart}
Usage FOGScheduler {start|stop|restart}
Script Completed Wed May 6 16:10:25 CDT 2015
Your logfile can be found in /home/jhales/test/FOGtroubleshoot.log

[url=“/_imported_xf_attachments/1/1977_FOGtroubleshoot.log.txt?:”]FOGtroubleshoot.log.txt[/url]