4 Tips for Writing Unsurprising Bash Scripts

Helpful guidelines for writing robust Bash scripts

Even though the first version of Bash was released over 30 years ago, people are still publishing best practice guides for writing Bash scripts (even Google has a bash style guide). If you’re just starting out with Bash, there are no shortage of good options out there (I heartily recommend this excellent cheat sheet).

As a consultant, I frequently see Bash scripts written for different purposes and environments, and after various stages of knowledge decay.

I want to share the guidelines that I’ve found most helpful when writing robust Bash scripts that run cross-platform and are pleasant to revisit months (or years) later. I’ve included commentary on mistakes I’ve made so that you don’t make them yourself.

There are four main tips:

  • Tell Bash to run in safe mode
  • Try to use the long-form of options
  • Use quotes around variables
  • Don’t write programs

Let’s go through each one in detail.

1) Tell Bash to Run in Safe Mode

Both of these headers tell Bash to exit immediately if:

  • a command fails (-e or -o errexit)
  • undefined variables are used (-u or -o nounset)

You’ve probably seen this before (especially if you followed the links in the introduction), so here are some common questions:

How do I set a variable to empty?

With nounset, you can use ${VARIABLE_NAME:-}, which will safely be empty if the variable is not set. You can use this in a test like so:

if [ -z "${VARIABLE_NAME:-}" ]; then
# $VARIABLE_NAME is empty
else
# $VARIABLE_NAME is not empty
fi

Should I use #!/bin/bash or #!/usr/bin/env bash?

Most guides will say that #!/usr/bin/env bash is more portable. In my experience, modern environments (even windows shells that can execute Bash) support #!/bin/bash.

I personally prefer the shorter version, because it includes the settings atomically on the first line. This means you can’t accidentally add code before errexit is set. But, I would accept either if I were reviewing code.

Can’t I do both?

Beware that you can’t use both techniques together:

# THIS IS BROKEN
#!/usr/bin/env bash -eu

This is because it will not set the options if someone tries to execute the script directly with bash ./your-script.sh, resulting in safe mode only working sometimes.

What about set -o pipefail?

A gotcha with set -o errexit is that it won’t exit if a command in a pipe fails:

# does not fail the script
some_failing_command | some_other_command

You can set -o pipefail to get around this, making the script fail if a command in a pipe fails.

# Will fail the script
set -o pipefail
some_failing_command | some_other_command

However, this not always the behaviour you want —for example, a grep which didn’t match anything returns a non-zero exit code, and would fail the script if used in a pipe.

Since pipefail isn’t always what you want, it’s hard to recommend a sensible default.

Some people recommend setting pipefail and putting || true on the lines you want to always pass.

# Will not fail the script
set -o pipefail
some_failing_command | some_other_command || true

The problem here is that failures in some_other_command are hidden, which is almost never what you want.

If you need a lot of pipes, think carefully about the failure cases. What do I recommend?

I recommend you don’t write too many pipes.

2) Try to Use the Long-Form of Options

Note that above we’re using the long-form of set -o errexit and set -o nounset rather than set -e and set -u, so that the meaning is clearer to future readers.

Most shell scripting environments (and many older programming languages) reward users for remembering arcane sequences of options. This can lead to code that is hard to maintain and create an unhelpful team culture, where people feel smart or well-studied instead of feeling like they’re great communicators. A team member saying “look how readable this is” is always nicer than “check out this clever one-liner”.

We can’t always use long-form options — for example, bash -o errexit only works with some bash versions, so we are forced to use the short-form if we want to ensure the safe-mode options are set at the very beginning of the script.

3) Use Quotes Around Variables

In my experience the most common cause of surprise script failure is spaces in pathnames:

# Don't do this
cp $FILE new-location# Do this
cp "$FILE" new-location

In the first case, spaces in the file name would expand the copy to a three (or more) argument command. The second case works as expected.

Ideally, your script contains no opinions on the directory or file names, and quoting your arguments means that your script doesn’t bake in the assumption that filenames don’t contain spaces.

I think this is the most important style habit, but of course, there are many other good opinions on Bash style. If you want deeper linting on your scripts, I recommend the excellent shellcheck utility.

4) Don’t Write Programs

Bash is very powerful. You can write reusable libraries of functions (even though Bash has no return values from functions, there are workarounds you can use), and you can create complex logic that interrogates the user’s environment with heavy levels of automation.

Don’t.

Google’s style guide recommends rewriting your script in Python if it is over 100 lines and I think this is an excellent guideline.

Bash doesn’t lend itself to clear, well-separated code, especially as your logic gets complex. It can be arcane — if you had to look it up whether to use $@ or $# while writing the script, so will some future readers. Critically, there’s no easy way to write tests for it.

I would also recommend not to have too many if/else branches (especially if they’re becoming nested), and not to use more than one or two functions.

If your script starts to do a lot of things, it’s probably time for it to be written in a language designed for doing a lot of things.

Special Circumstances

When to change these guidelines?

The best thing you can do is pay attention to the pain points when returning to your own scripts after many months away from them (or when using other people’s scripts for the first time).

When to break these guidelines?

My advice (for any set of guidelines) is to follow them unless you have a good reason not to. This means you get to be the judge of what “a good reason” means for you and your team.

For me, one common reason not to stick rigidly to these guidelines is when doing error handling — see the next post for details of guideline #5: Give helpful error messages.