Bulletproof Bash : Stop script on error

The other day I’ve been puzzled by the results of a cron job script.  The bash script in question was written in a hurry a while back, and I was under the assumption that if any of its steps fail, the whole script will fail.  I was wrong.  Some commands were failing, but the script execution continued.  It was especially difficult to notice, due to a number of unset variables, piped commands, and redirected error output.

Once I realized the problem, I got even more puzzled as to what was the best solution.  Sure, you can check an exit code after each command in the script, but that didn’t seem elegant of efficient.

A quick couple of Google searches brought me to this StackOverflow thread (no surprise there), which opened my eyes on a few bash options that can be set at the beginning of the script to stop execution when an error or warning occurs (similar to use strict; use warnings; in Perl).  Here’s the test script for you with some test commands, pipes, error redirects, and options to control all that.

#!/bin/bash

# Stop on error
set -e
# Stop on unitialized variables
set -u
# Stop on failed pipes
set -o pipefail

# Good command
echo "We start here ..."

# Use of non-initialized variable
echo "$FOOBAR"
echo "Still going after uninitialized variable ..."

# Bad command with no STDERR
cd /foobar 2> /dev/null
echo "Still going after a bad command ..."

# Good command into a bad pipe with no STDERR
echo "Good" | /some/bad/script 2> /dev/null
echo "Still going after a bad pipe ..."

# Benchmark
echo "We should never get here!"

Save it to test.sh, make executable (chmod +x test.sh), and run like so:

$ ./test.sh || echo Something went wrong

Then try to comment out some options and some commands to see what happens in different scenarios.

I think, from now on, those three options will be the standard way I start all of my bash scripts.

 

What are some things you wish you knew when you started programming?

Quora thread on “What are some things you wish you knew when you started programming?” is a goldmine of wisdom.  Irrelevant of how experienced you – whether you’ve been programming for decades or just thinking about a new career path, which programming languages and technology stacks you use, whether you’ve completed format education or taught yourself everything you know, I’m sure you’ll find valuable lessons and food for thought in there.

How to monitor your Linux servers with nmon

How to monitor your Linux servers with nmon” article provides some details on how to use the comprehensive server monitoring tool “nmon” (Nigel’s Monitor) to keep an eye on your server or two.  If you have more than a handful of servers, you’d probably opt out for a full blown monitoring solution, like Zabbix, but even with that, nmon can be useful for quick troubleshooting, screenshots, and data collection.

I’ve heard of nmon before and even used it occasionally.  What I didn’t know was that it can collect system metrics into a file, which can then later be analyzed and graphed with the nmonchart tool.

That’s pretty handy.  The extra bonus is that these tools are available in most Linux distributions, so there is no need to download/compile/configure things.

 

Nginx Amplify : comprehensive Nginx monitoring

Somehow I missed the announcement of the Nginx Amplify (beta) back in November of last year, so here it goes now.

Nginx Amplify is a new tool for the comprehensive monitoring of Nginx web servers.  Here’s what it can do for you:

  • Visually identify performance bottlenecks, overloaded servers, or potential DDoS attacks
  • Improve and optimize NGINX performance with intelligent advice and recommendations
  • Get alerts when something is wrong with the delivery of your application
  • Plan capacity and performance for web applications
  • Keep track of systems running NGINX

As well

as the regular proactive monitoring of the Nginx issues.  Have a look at the documentation for more details.