TS-32: Bash

🚧 DRAFT

This technical standard extends TS-31 with guidelines for using the Bash shell specifically. Everything in TS-31 applies here, too; TS-32 extends TS-31.

See also TS-62 for guidance on Makefiles, which commonly wrap Bash scripts and recipes covered by this standard.

Choosing Bash

Bash is the RECOMMENDED shell for scripts that will be used only internally within an organization. It is RECOMMENDED to standardize on Bash for all Unix scripting across an organization, to promote code reuse.

For scripts that will be distributed outside of an organization, or otherwise to environments in which the shell is unknown or can’t be controlled, it is RECOMMENDED to stick to POSIX-compliant syntax – as covered by the TS-31 technical standard.

File name extensions

Bash file names may have the .sh extension or no extension. Bash file names MUST NOT have other extensions such as .bash. Instead, use the shebang (see below) to identify the target interpreter.

Shebang (#!)

The first line of a Bash file MUST be the shebang (#!, also known as a hashbang). This indicates to the shell which interpreter to use. The classic Bash shebang is:

#!/bin/bash

But the env-style shebang is RECOMMENDED:

#!/usr/bin/env bash

The difference between the two is that the classic shebang bets everything on one path, while the env shebang looks for the Bash binary in more places, and thus the script has a greater chance of being successfully run across different environments. The trade-off is ever-so-slightly slower startup time, as the PATH environment variable needs to be searched to find the Bash binary.

Tip

If you’re using the classic Bash shebang, and if you get an error similar to "command not found" when running the script, then almost certainly the path to the Bash interpreter is wrong for the current system. Use whereis bash or find ./ -name bash to find the path to the bash executable. Some possible locations include:

  • /bin/bash
  • /sbin/bash
  • /usr/local/bin/bash
  • /usr/bin/bash
  • /usr/sbin/bash
  • /usr/local/sbin/bash

If a script is written to be POSIX-compliant, use either of the following shebangs instead. This will trip the Bash program to executing the script in POSIX-compliant mode, equivalent to passing the --posix option to bash on the command line (which in turn is equivalent to using the sh command to run Bash in POSIX-compliant mode).

#!/bin/sh
#!/bin/env sh

Again, the second env-style shebang will have wider compatibility and is therefore RECOMMENDED.

Options

It is RECOMMENDED to enable "strict mode" in all Bash scripts, by including the following line after the shebang:

set -euo pipefail

The -o pipefail option is a Bashism that extends POSIX strict mode (-e) to pipelines. It causes a pipeline to return a failure status if any command in the pipeline fails, rather than just the last command. This is useful for catching errors in pipelines that would otherwise pass by uncaught by the set -e exit mode.

The -e, -u, and -o pipefail options, used in combination, mean that the script will exit whenever any command returns a non-zero exit code (indicating an error), even if the command is executed within a pipeline, and when unset variables are encountered. This configuration is RECOMMENDED for most Bash script use cases, as it will help catch many common classes of bugs.

Sourcing

Use the POSIX-compliant dot notation for sourcing files. The alternative source keyword is more explicit but less portable (it is a Bash extension).

# ❌
source functions.sh

# ✅
. functions.sh

In Bash, sourced files are not truly modular. Sourcing merely provides a way of breaking up long scripts into smaller chunks, for easier code management. When sourcing a file, all Bash does is copy the contents of the sourced file and implant them into the current script at runtime.

Bash authors need to be careful about doing things like change directory (cd) within a child script, as doing so may break the relative paths for subsequent imports in the parent script. Best practice is for every sourced file to return the session to its pre-existing state. For example, if a source file uses cd, be sure to have the script cd back to the prior working directory before execution passes back to the parent script.

Bash authors also must be careful about mutating variables, declared in the parent script, in sourced scripts. Any variables that are defined in the parent script and referenced in the child script are handled as though all the code had been written in a single file. This is a common source of bugs – when two distinct values are accidentally assigned to variables of the same name, but declared in different files.

A good solution is to pass variables to source files, like this:

hello="helloworld"
. install.sh $hello

In install.sh:

# Prints "helloworld".
echo $1

The ability to source a script with arguments is a Bashism. In other shells such as Dash, positional parameters like $1 parameters will not be defined in this case.

Variables

In Bash, local variables can be used to scope data to the functions the variables are declared in.

hello="hello"

hello() {
  local hello="world"
  echo $hello # → "world"
}

echo $hello # → "hello" (not modified).

This syntax is Bash-specific and it is RECOMMENDED to use it for all variable declarations within functions, so as not to unnecessarily pollute the global namespace and to prevent unexpected overwrites of global variables. Pay particular attention to declaring loop variables as local, too – this is a common source of bugs:

local i
for i in {1..5}; do
  echo $i
done

Where the local keyword is omitted, include a comment explaining why it is not used. There will be many use cases where functions are designed to update global variables – that’s fine, just be sure to leave a comment to be explicit in your intent.

Variable declaration and assignment MAY be on different lines, but try to always initialize variables with meaningful default values.

some_function() {
  local code=0
  # …
  code=2

  return ${code}
}

Arrays

Arrays, which are a Bash-specific shell extension, are used to store lists of elements. This data structure is RECOMMENDED for safely expanding lists into individual elements, such as arguments given to a command or elements given to a loop. Arrays are perfect for these use cases – this is their intended purpose in the language design.

Arrays SHOULD NOT be used as a basic for constructing complex data structures – this is not their intended use case.

declare -a flags

# ✅
flags=(--foo --bar='baz')           # Initial assignment of flags.
flags+=(--greeting="Hello ${name}") # Append to the list of flags.
mybinary "${flags[@]}"              # Expand the flags to an arguments list.

# ❌ This won't work as expected.
flags='--foo --bar=baz'
flags+=' --greeting="Hello world"'
mybinary ${flags}

The names of variables that hold array structures SHOULD be pluralized. In loops, the singular form of the array name SHOULD be used for the iteration variable.

for zone in "${zones[@]}"; do
  # Do something with "${zone}".
done

Functions

Prefer the POSIX-compliant function declaration syntax. It is cleaner, more readable, and more portable.

# ❌ Explicit function declaration.
function some_function () {
  return 0
}

# ✅ Implicit function declaration.
some_function() {
  return 0
}

Returning values via namerefs

As covered in TS-31, the RECOMMENDED way to return non-integer data from a function is to write it to stdout and capture it with command substitution. That approach runs the function in a subshell, which has a performance cost and means the function cannot set a variable directly in the caller’s scope.

Where that overhead matters, or where a function needs to populate a variable in the caller’s environment directly – for example, an array or another structured value that would be awkward to serialize through stdout – use a Bash nameref (declare -n). The caller passes the name of a variable it owns as an argument, and the function assigns to that name as if it were a local alias for the caller’s variable:

get_config_value() {
  local -n result_ref="$1"
  local key="$2"
  result_ref="$(grep "^${key}=" config.env | cut -d '=' -f2-)"
}

get_config_value value "API_KEY"
echo "${value}"

Namerefs avoid the subshell entirely, so they are useful in loops or performance-sensitive code paths where repeated command substitution would be costly. They are a Bash-specific extension and MUST NOT be used in scripts that need to remain POSIX-compliant.

Control flows

For conditional expressions, it is RECOMMENDED to use the Bash-specific double-bracket syntax, [[ … ]], over [ … ], test, and /usr/bin/[.

In older versions of Bash, using single bracket syntax with && or || could cause syntax issues. Using the double bracket syntax is better, therefore, for backwards compatibility with older implementations of Bash.

The double bracket syntax is preferred for other reasons, too. It prevents pathname expansion and word splitting, which eliminates a common class of bugs in shell scripts. It also allows for regular expression matching, which the single bracket syntax does not support.

if [[ "filename" =~ ^[[:alnum:]]+name ]]; then
  echo "Match"
fi

if [[ "filename" == "f*" ]]; then
  echo "Match"
fi

# For comparison, this gives a "too many arguments" error as
# f* is expanded to the contents of the current directory.
if [ "filename" == f* ]; then
  echo "Match"
fi
Examples

For clarity, use == for equality rather than =, even though both work. The former requires the use of the preferred Bash-specific [[ … ]] syntax. The latter can be confused with an assignment.

#  ✅
if [[ "${my_var}" == "val" ]]; then
  do_something
fi

# ❌
if [[ "${my_var}" = "val" ]]; then
  do_something
fi

Be careful when using < and > in [[ … ]], which performs a lexicographical comparison. Use … or -lt and -gt for numerical comparison.

#  ✅
if (( my_var > 3 )); then
  do_something
fi

#  ✅
if [[ "${my_var}" -gt 3 ]]; then
  do_something
fi

# ❌ Probably unintended lexicographical comparison.
# True for 4, false for 22.
if [[ "${my_var}" > 3 ]]; then
  do_something
fi

Prefer to use -z and -n to test for zero-length and non-empty strings respectively. Alternatively you can do an equality check against a literal "" value, but if you do ensure that you quote on the empty side.

#  ✅
if [[ -z "${my_var}" ]]; then
  do_something
fi

# ✅
if [[ -n "${my_var}" ]]; then
  do_something
fi

# ✅ This is okay, but ensure quotes on the empty side.
if [[ "${my_var}" == "" ]]; then
  do_something
fi

# ❌ Be explicit and use `-n` here.
if [[ "${my_var}" ]]; then
  do_something
fi

# ❌ Do not use filler characters, like this.
if [[ "${my_var}X" == "some_stringX" ]]; then
  do_something
fi

Be careful about porting Bash scripts, which use the double-bracket [[ … ]] syntax, to other shells. Other shells have adopted this syntax, too, but the behavior is not consistent across all of them. Thus, the behavior of a script using this syntax could be inconsistent if executed in different shells.

In loops, Bash supports omitting the in "$@" part of a for loop. But it is RECOMMENDED to maintain this, for clarity.

for arg in "$@"; do
  echo "${arg}"
done

Process substitution

Process substitution, <(command) and >(command), lets the output (or input) of a command be treated as if it were a file. It is RECOMMENDED as a lightweight alternative to mktemp for one-off cases where a command’s output only needs to be read once, avoiding the need to create and clean up a temporary file.

# ❌ Requires creating and cleaning up a temporary file.
tmp="$(mktemp)"
first-stage > "${tmp}"
second-stage --input "${tmp}"
rm -f "${tmp}"

# ✅ No temporary file required.
second-stage --input <(first-stage)

Prefer mktemp over process substitution where the same output needs to be read multiple times, where the consuming command does not support reading from a named pipe, or where the file needs to persist beyond the current command.

Extended globbing

The shopt -s extglob and shopt -s globstar options extend Bash’s pattern matching. extglob enables extended pattern operators such as @(pattern-list) and !(pattern-list), allowing more expressive filename matching within the shell itself. globstar enables ** to match files recursively through subdirectories.

shopt -s extglob globstar

# Match any file ending in .adoc, at any depth.
files=(**/*.adoc)

# Match files matching either of two patterns.
items=(**/@(foo|bar)*)

These options are RECOMMENDED where they can replace an external find invocation for simple recursive or pattern-based file matching, avoiding the overhead of spawning a subprocess. For more complex matching – conditions on file type, permissions, modification time, and so on – find remains the appropriate tool.

Arithmetic

Arithmetic expressions MUST be written using … or $…, and MUST NOT use the $[ … ] syntax, the `expr command, or the let built-in.

# ❌ This Bash syntax is deprecated and non-portable.
i=$[2 * 10]

# ❌ Unquoted assignments using `let` are subject to globbing word-splitting.
let i="2 + 2"

# ❌ The expr utility is an external program, not a shell built-in, and is
# many times slower than the built-in `(( … ))` syntax for arithmetic. There
# are other issues with using `expr` too, such as inconsistencies with the
# handling of quoting.
i=$( expr 4 '*' 4 )

Numerical comparison MUST NOT be performed inside [[ … ]] expressions, either. In this context, the < and > operators perform lexicographical comparison instead. For all numeric comparisons use ` …`.

if (( a < b )); then
  …
fi

Variables MAY be referenced verbatim inside $…. Bash knows to look up var for you; you don’t need to write ${var} or even $var. Referencing variables consistently throughout a script is RECOMMENDED, but inside $… is the one exception where you MAY omit the ${…} wrapping syntax. This produces slightly cleaner code and tends to read better in the context of arithmetic expressions.

hr=2
min=5
sec=30
echo "$(( hr * 3600 + min * 60 + sec ))" # → 7530

In the following example, note the use of $… within a string.

echo "$(( 2 + 2 )) is 4"

It is recommended to avoid using … as a standalone statement. But it’s okay sometimes. In the following example, the result of a calculation is assigned to a variable.

(( i = 10 * j + 400 ))

References