Getting Started with Bash Testing with Bats

This article has been published a while ago.

If this is a technical article some information might be out of date. If something is terribly broken, let me know and I will update the article accordingly.

In this guide I would like to give you an introduction on how you can start writing tests for your Bash scripts by using Bats.

Note: I'm by no means an expert when it comes to writing tests in Bash. All my knowledge comes from maintaining a GitHub Action called git-auto-commit. While I've developed the tests for git-auto-commit, I wished there was a resource which would guide me through all the steps necessary to add a test suite to my bash scripts. That's why this guide exists.

The source code referenced in this guide can be found in this GitHub repository.

Prerequisites

This guide assume that you have basic knowledge of using the shell and about testing code in general. You should also have the npm package manger installed on your machine.
In the later parts of this guide, I also assume that you use Git and host your code on GitHub.

Why and when should you write tests for shell scripts

Writing tests for your projects should give you the confidence that your piece of code does what you expect it do to. Yes, it will take a bit more time to finish the project, but your future self – or another person – will be thankful for those tests.

But writing tests for every single line of code probably doesn't make sense. So my general rule of thumb is as follows:

If it's a simple one-off script you will only run once it's probably overkill to write tests for it.
If the scripts is shared with the public, should work for future software versions, I will maintain it for the foreseeable future and it isn't a one-liner, it's probably worth investing the time to write a test suite.

If the second statement speaks to you, please read further.

Add Bats to your project

The easiest way to install Bats is by using the npm package manager.
Counterintuitive, right? Why would you use a JavaScript package manager to install a Bash utility?

The answer is simple: npm is a very popular tool among developers. (I would say most developers have it installed on their machine.)
In addition, it comes preinstalled on most continous integration environments. This makes testing your script on a CI service like GitHub Actions much easier. And last, it streamlines the setup process for contributors. They don't have to go through a 10 step install-instructions manual just to write tests. All they have to do is run npm install and they are good to go.

So let's add an empty package.json file to our project by running the following command.

echo "{}" > package.json

Now we can install Bats.

npm install --save-dev bats

To make running the tests easier, we're also adding a scripts block to the package.json file. This allows us to execute our test suite by running npm run test.

 {
   "devDependencies": {
     "bats": "^1.1.0"
   },
 "scripts": {
   "test": "bats tests"
 }
 }

What's missing now is our test file. Let's create it by running:

touch tests/my-first-test.bats

Add the following code inside tests/my-first-test.bats. This is how our very first test looks like:

#!/usr/bin/env bats

@test "It Works" {
    # Arrange
    # Prepare "the world" for your test

    # Act
    # Run your code
    result="$(echo 2+2 | bc)"

    # Assert
    # Make assertions to ensure that the code does what it should
    [ "$result" -eq 4 ]
}

If you're running npm run test in your console, you should see:

> test
> bats tests

 ✓ It Works

1 test, 0 failures

Now, let us install two additional libraries to make writing tests easier.

Install bats-asserts and bats-support

bats-assert and bats-support are two libraries which will make writing Bash tests much more intuitive.

bats-assert gives us an array of helper functions to write assertions that are much easier to process for us humans.

For example for me assert_line 'usage: foo <filename>' is much easier to read and understand than [ "${lines[0]}" = "usage: foo <filename>" ]. (Head over to the bats-assert repository to learn more about the available assertions).

(bats-support is just a supporting library used by bats-assert under the hood.)

To install both libraries run the following commands in your terminal:

npm install --save-dev https://github.com/ztombol/bats-support
npm install --save-dev https://github.com/ztombol/bats-assert

Your package.json file should now look like this.

 {
   "devDependencies": {
     "bats": "^1.1.0",
   "bats-assert": "ztombol/bats-assert",
   "bats-support": "ztombol/bats-support"
   },
   "scripts": {
     "test": "bats tests"
   }
 }

To make use of those two libraries we have to load them in our test file. Add the following lines at the top of my-first-test.bats.

#!/usr/bin/env bats

load '../node_modules/bats-support/load'
load '../node_modules/bats-assert/load'

# ...

You can now also update the assertion in our first test to use assert_equal.

# Assert
# Make assertions to ensure that the code does what it should
[ "$result" -eq 4 ]
assert_equal "$result" 4

If you now run npm run test you should still see that all tests pass.

So we have written a test for a Bash script. Great!
But it's very bare bone right now, right? Let's kick it up a notch.

Writing tests for your existing shell script

In our existing tests we are doing some basic math. The test is also not using code we have written outside of the test case itself.
In a real world project you probably have written your code in a file called main.sh or entrypoint.sh.

Let's update our test suite to run main.sh and then write assertions to make sure, it does what it should.

Let's create a main.sh script.

touch main.sh
chmod +x main.sh

Add the following code inside the newly created main.sh.

#!/bin/bash

touch ./file-{1,2,3}.txt

All our script does is create 3 txt-files in the current directory. Run ./main.sh to see it in action.

Now let's add a test for this. In our my-first-test.bats file add a new test case. I've added comments in the code below to explain what each line does.

# previous test cases …

@test "It creates 3 txt files" {
    # Delete possible existing txt-files
    rm -rf file-*.txt

    # Run our script.
    # We use $BATS_TEST_DIRNAME here, as the tests
    # are executed in a temporary directory. The
    # variable gives us the absolute path to
    # the testing directory
    run "${BATS_TEST_DIRNAME}"/../main.sh

    # Assert that the script has run successfully
    assert_success

    # Execute `ls` to return a list of
    # all the files in the directory
    run ls

    # Assert against the output of `ls` that
    #  file-1.txt, file-2.txt and file-3.txt exist
    assert_output --partial 'file-1.txt'
    assert_output --partial 'file-2.txt'
    assert_output --partial 'file-3.txt'
}

The test should be straightforward to follow:

We remove pre-existing txt-files to ensure, that we don't have any false positives
We run our main.sh script
We assert that the script has run successfully
We execute ls to get a list of all files in the current directory
We assert that the output of ls contains the names of our generated files.

Even though our script and tests are quite simple, this example covers the most common problems I faced while writing tests:

How can I execute my own script within a test?
How can I write assertions to check that my script works as expected?

In the next step we will add a setup and teardown function to clean the test up.

Add setup() and teardown() to clean up your test

The setup and teardown functions are a common pattern in testing frameworks. PHPUnit has them. Jest has them. And many more frameworks have them too.

These functions give you the ability to run a bit of code before and after a single test case is run. These hooks are commonly used to prepare the "world" for our tests. (See the Bats documentation for details)

Let us add these two hooks to our test suite.

#!/usr/bin/env bats

load '../node_modules/bats-support/load'
load '../node_modules/bats-assert/load'

setup() {
    # Add code which should be executed before each test case
    export MY_SCRIPT_PATH_FOR_NEW_FILES=.
}

teardown() {
    rm -rf ${MY_SCRIPT_PATH_FOR_NEW_FILES}/file-*.txt
}

# test cases …

Our setup function currently exports a MY_SCRIPT_PATH_FOR_NEW_FILES variable, which we will use in our script in a moment.
The teardown function removes all txt-files so that we don't have to repeat ourselfs in every test case.

I've also updated our main.sh script to use MY_SCRIPT_PATH_FOR_NEW_FILES.

#!/bin/bash

touch ${MY_SCRIPT_PATH_FOR_NEW_FILES}/file-{1,2,3}.txt

And our test case has been updated as well.

@test "It creates 3 txt files" {
    # Run our script.
    # We use $BATS_TEST_DIRNAME here, as the tests
    # are executed in a temporary directory. The
    # variable gives us the absolute path to
    # the testing directory
    run "${BATS_TEST_DIRNAME}"/../main.sh

    # Assert that the script has run successfully
    assert_success

    # Execute `ls` to return a list of
    # all the files in the directory
    run ls ${MY_SCRIPT_PATH_FOR_NEW_FILES}

    # Assert against the output of `ls` that
    #  file-1.txt, file-2.txt and file-3.txt exist
    assert_output --partial 'file-1.txt'
    assert_output --partial 'file-2.txt'
    assert_output --partial 'file-3.txt'
}

With this setup you can now create additional test cases to cover all possible edge cases for your script. In your setup function you can prepare your "world" for the tests and in teardown you can clean up your "world" again.

If you want to see a real world example I encourage you to take a look at the tests I've written for git-auto-commit. You can find them on GitHub.

Automatically run test suite on GitHub Actions

Now that we have our test suite ready, we want to make sure that our tests also run in a continuous integration environment.
Or in other words: Let us set up GitHub Actions to run our tests whenever we or a contributor pushes code to the repository or opens a pull request.

If you haven't done already, create a new repository on GitHub for your script. Clone the repository to your machine, add all project files to the repository, create a git commit and push the code to GitHub.

Next, create .github/workflows/test.yml. In this file, we will add our Workflow configuration to run our tests on GitHub Actions.

touch .github/workflows/test.yml

Add the following lines to test.yml.

name: tests

on: push

jobs:
    tests:
        runs-on: ubuntu-latest

        steps:
            - uses: actions/checkout@v2

            - name: Install dependencies
              run: npm install

            - name: Run Tests
              run: npm run test

A quick summary, what this Workflow does.

On each git commit push the Workflow is run
The repository is checked out
Our dependencies (bats, bats-assert and bats-support) are being installed with npm
Our test suite is run with npm run test

Now commit your changes and push them to the repository on GitHub. In a matter of seconds your GitHub Action run should start and your tests should run successfully and be "green".

Going further: Mocking with Shellmock

I won't add mocking capabilities to our example project here, but I would like to give a short overview on how you can add Mocking to your Bats tests.
For my git-auto-commit project I've used Shellmock for a period of time, to mock all calls to git. (In hindsight, that was a mistake. Full story can be read here).

First, you need to install Shellmock by running the following commands.

git clone https://github.com/capitalone/bash_shell_mock
cd bash_shell_mock
sudo ./install.sh /usr/local

(Read the projects README if this doesn't work)

Next we need to load – or "source" – Shellmock in our tests. Update setup and teardown like this:

setup() {
    . shellmock
}

teardown() {
    if [ -z "$TEST_FUNCTION" ]; then
        shellmock_clean
    fi
}

. shellmock will load/source Shellmock and make its functions available to us.
‌shellmock_clean will remove various temp files which will be created by Shellmock.

Now we are ready to mock calls to binaries our script is using. In this section of the guide the examples will cover calls to git (as it's the only example I could think of and which I've used in the past).

Imagine our main.sh script does the following.

#!/bin/bash

git add .
git commit -m "This is the way"
git push origin

It stages all changed files in the Git repository, will create a new commit and will push the changes to the remote repository.

In our tests, we don't want to actually create a commit and push them to the repository. It would make our tests rely on a Network connection and would make the tests slow. So we will mock calls to git.

A test case using Shellmock would look like this.

@test "It commits changed files" {
    # Create multiple mocks for git
    shellmock_expect git --type partial --match "add"
    shellmock_expect git --type partial --match 'commit'
    shellmock_expect git --type partial --match 'push origin'

    run "${BATS_TEST_DIRNAME}"/../main.sh

    # Assert that the calls to git happended
    shellmock_verify
    [ "${capture[0]}" = "git-stub add ." ]
    [ "${capture[1]}" = "git-stub commit -m 'This is the way'" ]
    [ "${capture[2]}" = "git-stub push origin" ]
}

In contrast to our other test cases in the rest of this guide, the test is getting quite noisy with the calls to shellmock_expect. The assertions are also not as readable as assert_line or assert_equals.
You also have to explicitly tell the order in which you expect each call to git is going to happen.

That's a lot of brain work. 🙃

In short: Try to avoid mocking calls to other binaries if possible. From experience I can tell you, that your tests will become brittle, because you have to update your tests whenever you update your script.

This will lead to a lot of frustration and wasted time.

Outro

If you've read so far: Congrats! You should now know the basics on how to tests Bash scripts with Bats. 🎉

As a reminder, you can find the source code of the examples mentioned in this guide on GitHub.

If you have questions or suggestions feel free to open a Discussion on the GitHub repository or contact me via Twitter or email.

Getting Started with Bash Testing with Bats

This article has been published a while ago.

Prerequisites#

Why and when should you write tests for shell scripts#

Add Bats to your project#

Install bats-asserts and bats-support#

Writing tests for your existing shell script#

Add setup() and teardown() to clean up your test#

Automatically run test suite on GitHub Actions#

Going further: Mocking with Shellmock#

Outro#