Linux terminal on a laptop screen over a blue background.
fatmawati achmad zaenuri/Shutterstock.com

The Linux set and pipefail commands dictate what happens when a failure occurs in a Bash script. There’s more to think about than should it stop or should it carry on.

RELATED: The Beginner's Guide to Shell Scripting: The Basics

Bash Scripts and Error Conditions

Bash shell scripts are great. They’re quick to write and they don’t need compiling. Any repetitive or multi-stage action that you need to perform can be wrapped in a convenient script. And because scripts can call any of the standard Linux utilities, you’re not limited to the capabilities of the shell language itself.

But problems can arise when you call an external utility or program. If it fails, the external utility will close down and send a return code to the shell, and it might even print an error message to the terminal. But your script will carry on processing. Perhaps that’s not what you wanted. If an error occurs early in the execution of the script it might lead to worse issues if the rest of the script is allowed to run.

You could check the return code from each external process as they complete, but that becomes difficult when processes are piped into other processes. The return code will be from the process at the end of the pipe, not the one in the middle that failed. Of course, errors can occur inside your script too, such as trying to access an uninitialized variable.

The set and pipefile commands let you decide what happens when errors like these occur. They also let you detect errors even when they happen in the middle of a pipe chain.

Here’s how to use them.

Demonstrating the Problem

Here’s a trivial Bash script. It echoes two lines of text to the terminal. You can run this script if you copy the text into an editor and save it as “script-1.sh.”

#!/bin/bash

echo This will happen first 
echo This will happen second

To make it executable you’ll need to use chmod:

chmod +x script-1.sh
Advertisement

You’ll need to run that command on each script if you want to run them on your computer. Let’s run the script:

./script-1.sh

Running a simple script with no errors.

The two lines of text are sent to the terminal window as expected.

Let’s modify the script slightly. We’ll ask ls to list the details of a file that doesn’t exist. This will fail. We saved this as “script-2.sh” and made it executable.

#!/bin/bash

echo This will happen first
ls imaginary-filename
echo This will happen second

When we run this script we see the error message from ls .

./script-2.sh

Running a script and generating a fail condition.

Although the ls command failed, the script continued to run. And even though there was an error during the script’s execution, the return code from the script to the shell is zero, which indicates success. We can check this using echo and the $? variable which holds the last return code sent to the shell.

echo $?

Checking the return code for the last script executed.

Advertisement

The zero that gets reported is the return code from the second echo in the script. So there are two issues with this scenario. The first is the script had an error but it carried on running. That can lead to other problems if the rest of the script expects or depends on the action that failed actually succeeded. And the second is that if another script or process needs to check the success or failure of this script, it’ll get a false reading.

The set -e Option

The set -e (exit) option causes a script to exit if any of the processes it calls generate a non-zero return code. Anything non-zero is taken to be a failure.

By adding the set -e option to the start of the script, we can change its behavior. This is “script-3.sh.”

#!/bin/bash 
set -e

echo This will happen first
ls imaginary-filename
echo This will happen second

If we run this script we’ll see the effect of set -e.

./script-3.sh
echo $?

Terminating a script upon an error condition, and correctly setting the return code.

The script is halted and the return code sent to the shell is a non-zero value.

Dealing With Failures in Pipes

Piping adds more complexity to the problem. The return code that comes out of a piped sequence of commands is the return code from the last command in the chain. If there’s a failure with a command in the middle of the chain we’re back to square one. That return code is lost, and the script will carry on processing.

Advertisement

We can see the effects of piping commands with different return codes using the true and false shell built-ins. These two commands do no more than generate a return code of zero or one, respectively.

true
echo $?
false
echo $?

The bash shell true and false built-in commands.

If we pipe false into true —with false representing a failing process—we get true‘s return code of zero.

false | true
echo $?

Piping false into true.

Bash does have an array variable called PIPESTATUS, and this captures all of the return codes from each program in the pipe chain.

false | true | false | true
echo "${PIPESTATUS[0]} ${PIPESTATUS[1]} ${PIPESTATUS[2]} ${PIPESTATUS[3]}"

Using PIPESTATUS to see the return code of all programs in a pipe chain.

PIPESTATUS only holds the return codes until the next program runs, and trying to determine which return code goes with which program can get very messy very quickly.

This is where set -o (options) and pipefail come in. This is “script-4.sh.” This will try to pipe the contents of a file that doesn’t exist into wc.

#!/bin/bash 
set -e

echo This will happen first
cat script-99.sh | wc -l
echo This will happen second

This fails, as we’d expect.

./script-4.sh
echo $?

Running a script with an error in a pipe chain.

The first zero is the output from wc, telling us it didn’t read any lines for the missing file. The second zero is the return code from the second echo command.

Advertisement

We’ll add in the -o pipefail , save it as “script-5.sh”, and make it executable.

#!/bin/bash 
set -eo pipefail

echo This will happen first
cat script-99.sh | wc -l
echo This will happen second

Let’s run that and check the return code.

./script-5.sh
echo $?

Running a script that traps errors in pipe chains and correctly sets the return code.

The script halts and the second echo command isn’t executed. The return code sent to the shell is one, correctly indicating a failure.

RELATED: How to Use the Echo Command on Linux

Catching Uninitialized Variables

Uninitialized variables can be difficult to spot in a real-world script. If we try to echo the value of an uninitialized variable, echo simply prints a blank line. It doesn’t raise an error message. The rest of the script will continue to execute.

This is script-6.sh.

#!/bin/bash 
set -eo pipefail

echo "$notset" 
echo "Another echo command"

We’ll run it and observe its behavior.

./script-6.sh
echo $?

Running a script that doesn't capture uninitialized variables.

The script steps over the uninitialized variable, and continues to execute. The return code is zero. Trying to find an error like this in a very long and complicated script can be very difficult.

We can trap this type of error using the set -u (unset) option. We’ll add that to our growing collection of set options at the top of the script, save it as “script-7.sh”, and make it executable.

#!/bin/bash 

set -eou pipefail

echo "$notset" 

echo "Another echo command"

Let’s run the script:

./script-7.sh
echo $?

Running a script that does capture uninitialized variables.

The uninitialized variable is detected, the script halts, and the return code is set to one.

Advertisement

The -u (unset) option is intelligent enough not to be triggered by situations where you can legitimately interact with an uninitialized variable.

In “script-8.sh”, the script checks whether the variable New_Var is initialized or not. You don’t want the script to stop here, in a real-world script you’ll perform further processing and deal with the situation yourself.

Note that we’ve added the -u option as the second option in the set statement. The -o pipefail option must come last.

#!/bin/bash 

set -euo pipefail

if [ -z "${New_Var:-}" ]; then 

echo "New_Var has no value assigned to it." 

fi

In “script-9.sh”, the uninitialized variable is tested and if it is uninitialized, a default value is provided instead.

#!/bin/bash
set -euo pipefail

default_value=484
Value=${New_Var:-$default_value}
echo "New_Var=$Value"

The scripts are allowed to run through to their completion.

./script-8.sh
./script-9.sh

Running two scripts where the uninitialized variables are handled internally, and the -u option doesn't trigger.

Sealed With a x

Another handy option to use is the set -x (execute and print) option. When you’re writing scripts, this is can be a lifesaver. it prints the commands and their parameters as they are executed.

Advertisement

It gives you a quick “rough and ready” form of execution trace. Isolating logic flaws and spotting bugs becomes much, much easier.

We’ll add the set -x option to “script-8.sh”, save it as “script-10.sh”, and make it executable.

#!/bin/bash
set -euxo pipefail

if [ -z "${New_Var:-}" ]; then
  echo "New_Var has no value assigned to it."
fi

Run it to see the trace lines.

./script-10.sh

Running a script with -x trace lines written to the terminal.

Spotting bugs in these trivial example scripts is easy. When you start to write more involved scripts, these options will prove their worth.

Profile Photo for Dave McKay Dave McKay
Dave McKay first used computers when punched paper tape was in vogue, and he has been programming ever since. After over 30 years in the IT industry, he is now a full-time technology journalist. During his career, he has worked as a freelance programmer, manager of an international software development team, an IT services project manager, and, most recently, as a Data Protection Officer. His writing has been published by  howtogeek.com, cloudsavvyit.com, itenterpriser.com, and opensource.com. Dave is a Linux evangelist and open source advocate.
Read Full Bio »