Importing functions from a shell script

Question

I have a shell script that I would like to test with shUnit. The script (and all the functions) are in a single file since it makes installation much easier.

Example for script.sh

#!/bin/sh

foo () { ... }
bar () { ... }

code

I wanted to write a second file (that does not need to be distributed and installed) to test the functions defined in script.sh

Something like run_tests.sh

#!/bin/sh

. script.sh

# Unit tests

Now the problem lies in the . (or source in Bash). It does not only parse function definitions but also executes the code in the script.

Since the script with no arguments does nothing bad I could

. script.sh > /dev/null 2>&1

but I was wondering if there is a better way to achieve my goal.

Edit

My proposed workaround does not work in the case the sourced script calls exit so I have to trap the exit

#!/bin/sh

trap run_tests ERR EXIT

run_tests() {
   ...
}

. script.sh

The run_tests function is called but as soon as I redirect the output of the source command the functions in the script are not parsed and are not available in the trap handler

This works but I get the output of script.sh:

#!/bin/sh
trap run_tests ERR EXIT
run_tests() {
   function_defined_in_script_sh
}
. script.sh

This does not print the output but I get an error that the function is not defined:

#!/bin/sh
trap run_tests ERR EXIT
run_tests() {
   function_defined_in_script_sh
}
. script.sh | grep OUTPUT_THAT_DOES_NOT_EXISTS

This does not print the output and the run_tests trap handler is not called at all:

#!/bin/sh
trap run_tests ERR EXIT
run_tests() {
   function_defined_in_script_sh
}
. script.sh > /dev/null

Suppliant

See also: What is the bash equivalent to Python's if __name__ == '__main__'?. I've added a very Python-like answer there for anyone who's interested. – Tanga

Answer 1

According to the “Shell Builtin Commands” section of the bash manpage, . aka source takes an optional list of arguments which are passed to the script being sourced. You could use that to introduce a do-nothing option. For example, script.sh could be:

#!/bin/sh

foo() {
    echo foo $1
}

main() {
    foo 1
    foo 2
}

if [ "${1}" != "--source-only" ]; then
    main "${@}"
fi

and unit.sh could be:

#!/bin/bash

. ./script.sh --source-only

foo 3

Then script.sh will behave normally, and unit.sh will have access to all the functions from script.sh but will not invoke the main() code.

Note that the extra arguments to source are not in POSIX, so /bin/sh might not handle it—hence the #!/bin/bash at the start of unit.sh.

Tompion

You might want to shift the argument list so main does not get in contact with the --source-only – Belshazzar

@HubertGrzeskowiak Good point, fixed. Thanks for the suggestion! – Tompion

The shift doesn't make sense there, because the main only runs if --source-only is not the first argument. – Teutonism

@HelderPereira Good catch! Thanks for reporting that. – Tompion

My answer I just added just magically works, just like in Python. No need for the extra --source-only argument. See also my explanation in my other answer here: What is the bash equivalent to Python's if __name__ == '__main__'?. I've also verified that my technique works even in nested scripts, where one bash script calls or sources another, which calls or sources another. – Tanga

Answer 2

Picked up this technique from Python, but the concept works just fine in bash or any other shell...

The idea is that we turn the main code section of our script into a function. Then at the very end of the script, we put an 'if' statement that will only call that function if we executed the script but not if we sourced it. Then we explicitly call the script() function from our 'runtests' script which has sourced the 'script' script and thus contains all its functions.

This relies on the fact that if we source the script, the bash-maintained environment variable $0, which is the name of the script being executed, will be the name of the calling (parent) script (runtests in this case), not the sourced script.

(I've renamed script.sh to just script cause the .sh is redundant and confuses me. :-)

Below are the two scripts. Some notes...

• $@ evaluates to all of the arguments passed to the function or script as individual strings. If instead, we used $*, all the arguments would be concatenated together into one string.
• The RUNNING="$(basename $0)" is required since $0 always includes at least the current directory prefix as in ./script.
• The test if [[ "$RUNNING" == "script" ]].... is the magic that causes script to call the script() function only if script was run directly from the commandline.

script

#!/bin/bash

foo ()    { echo "foo()"; }

bar ()    { echo "bar()"; }

script () {
  ARG1=$1
  ARG2=$2
  #
  echo "Running '$RUNNING'..."
  echo "script() - all args:  $@"
  echo "script() -     ARG1:  $ARG1"
  echo "script() -     ARG2:  $ARG2"
  #
  foo
  bar
}

RUNNING="$(basename $0)"

if [[ "$RUNNING" == "script" ]]
then
  script "$@"
fi

runtests

#!/bin/bash

source script 

# execute 'script' function in sourced file 'script'
script arg1 arg2 arg3

Strega

This has two downsides: whenever you rename the script, you have to change its contents; and it doesn't work with symlinks (not sure about aliases). – Belshazzar

It throws an error if executed from the command line: basename: illegal option -- b. – Choroid

Answer 3

If you are using Bash, a similar solution to @andrewdotn's approach (but without needing an extra flag or depending on the script name) can be accomplished by using BASH_SOURCE array.

script.sh:

#!/bin/bash

foo () { ... }
bar () { ... }

main() {
    code
}

if [[ "${#BASH_SOURCE[@]}" -eq 1 ]]; then
    main "$@"
fi

run_tests.sh:

#!/bin/bash

. script.sh

# Unit tests

Teutonism

That's pretty neat! I'd personally exchange the test condition to (( ${#BASH_SOURCE[@]} == 1 )); the short conditional (( ${#BASH_SOURCE[@]} == 1 )) && main "$@" should work, too, and suffices if you don't want it to do anything else. – Appellant

Answer 4

Now the problem lies in the . (or source in Bash). It does not only parse function definitions but also executes the code in the script.

Here is a way I like to ensure my Bash code is only run if the file is executed, not sourced.

I like to make it look even more Python-like than the other answers here. I do a little extra so I can end up with the if [ "$__name__" = "__main__" ]; then line. See also my answer here: What is the bash equivalent to Python's if __name__ == '__main__'?, where I describe this:

if [ "${BASH_SOURCE[0]}" = "$0" ]; then
    # This script is being run.
    __name__="__main__"
else
    # This script is being sourced.
    __name__="__source__"
fi

# Code entry point. Only run `main` if this script is being **run**, NOT
# sourced (imported).
# - See my answer: https://mcmap.net/q/12593/-what-is-the-bash-equivalent-to-python-39-s-if-__name__-39-__main__-39
if [ "$__name__" = "__main__" ]; then
    main "$@"
fi

Let's take this further though, and I'll present a full library example and a variety of ways to import it:

Detailed example: how do you write, import, use, and test libraries in Bash?

Here's a really beautiful, almost Python-like style I've come with over the years for writing and using Bash libraries. Bash is a beautiful "glue"-type language which allows you to easily tie together executables from many languages. Considering how long Bash has been around, I'm not sure why the below style isn't more popular, but perhaps it hasn't been thought of or used this way before. So, here goes. I think you'll find it really useful.

You can also see a general starting point I use for all of my bash scripts in my hello_world_best.sh file in my eRCaGuy_hello_world repo.

You can see a full library example in floating_point_math.sh.

library_basic_example.sh:

#!/usr/bin/env bash

RETURN_CODE_SUCCESS=0
RETURN_CODE_ERROR=1

# Add your library functions here. Ex:

my_func1() {
    echo "100.1"
}

my_func2() {
    echo "200"
}

my_func3() {
    echo "hello world"
}

# Note: make "private" functions begin with an underscore `_`, like in Python,
# so that users know they are not intended for use outside this library.

# Assert that the two input argument strings are equal, and exit if they are not
_assert_eq() {
    if [ "$1" != "$2" ]; then
        echo "Error: assertion failed. Arguments not equal!"
        echo "  arg1 = $1; arg2 = $2"
        echo "Exiting."
        exit $RETURN_CODE_ERROR
    fi
}

# Run some unit tests of the functions found herein
_test() {
    printf "%s\n\n" "Running tests."

    printf "%s\n" "Running 'my_func1'"
    result="$(my_func1)"
    printf "%s\n\n" "result = $result"
    _assert_eq "$result" "100.1"

    printf "%s\n" "Running 'my_func2'"
    result="$(my_func2)"
    printf "%s\n\n" "result = $result"
    _assert_eq "$result" "200"

    printf "%s\n" "Running 'my_func3'"
    result="$(my_func3)"
    printf "%s\n\n" "result = $result"
    _assert_eq "$result" "hello world"

    echo "All tests passed!"
}

main() {
    _test
}

# Determine if the script is being sourced or executed (run).
# See:
# 1. "eRCaGuy_hello_world/bash/if__name__==__main___check_if_sourced_or_executed_best.sh"
# 1. My answer: https://mcmap.net/q/12593/-what-is-the-bash-equivalent-to-python-39-s-if-__name__-39-__main__-39
if [ "${BASH_SOURCE[0]}" = "$0" ]; then
    # This script is being run.
    __name__="__main__"
else
    # This script is being sourced.
    __name__="__source__"
fi

# Code entry point. Only run `main` if this script is being **run**, NOT
# sourced (imported).
# - See my answer: https://mcmap.net/q/12593/-what-is-the-bash-equivalent-to-python-39-s-if-__name__-39-__main__-39
if [ "$__name__" = "__main__" ]; then
    main "$@"
fi

Run the library to run its unit tests

Now, make the file executable. Running it will run its internal unit tests:

# make it executable
chmod +x library_basic_example.sh

# run it
./library_basic_example.sh

Example run command and output:

eRCaGuy_hello_world$ bash/library_basic_example.sh 
Running tests.

Running 'my_func1'
result = 100.1

Running 'my_func2'
result = 200

Running 'my_func3'
result = hello world

All tests passed!

Import (source) the library

To import a Bash library, you "source" it with the source or . (better) command. Read more about that in my answer here: source (.) vs export (and also some file lock [flock] stuff at the end).

1. Use a manually-set import path

You can do this either in a bash terminal directly, or in your own bash script. Try it right now inside your own terminal!:

source "path/to/library_basic_example.sh"

# Or (better, since it's Posix-compliant)
. "path/to/library_basic_example.sh"

Once you source (import) the Bash library, you get access to call its functions directly. Here's a full example run and output, showing that I can do function calls like my_func1, my_func2, etc, right inside my terminal once I've sourced (imported) this Bash library!:

eRCaGuy_hello_world$ . bash/library_basic_example.sh
eRCaGuy_hello_world$ my_func1
100.1
eRCaGuy_hello_world$ my_func2
200
eRCaGuy_hello_world$ my_func3
hello world
eRCaGuy_hello_world$ _test
Running tests.

Running 'my_func1'
result = 100.1

Running 'my_func2'
result = 200

Running 'my_func3'
result = hello world

All tests passed!

2. Use a BASHLIBS environment variable to make importing your Bash libraries easier

You can source your bash libraries from any path. But, an environment variable, such as BASHLIBS, makes it easier. Add this to the bottom of your ~/.bashrc file:

if [ -d "$HOME/libs_bash/libraries" ] ; then
    export BASHLIBS="$HOME/libs_bash/libraries"
fi

Now, you can symlink your Bash libraries into that directory, like this:

# symlink my `library_basic_example.sh` file into the `~/libs_bash/libraries`
# dir
mkdir -p ~/libs_bash/libraries
cd path/to/dir_where_my_library_file_of_interest_is_stored
# make the symlink
ln -si "$(pwd)/library_basic_example.sh" ~/libs_bash/libraries/

Now that a symlink to my library_basic_example.sh file is stored inside ~/libs_bash/libraries/, and the BASHLIBS environment variable has been set and exportted to my environment, I can import my library into any Bash terminal or script I run within the terminal, like this:

. "$BASHLIBS/library_basic_example.sh"

3. [My most-used technique!] Using relative import paths

This is really beautiful and powerful. Check this out!

Let's say you have the following directory layout:

dir1/
    my_lib_1.sh

    dir2/
        my_lib_2.sh
        my_script.sh
    
        dir3/
            my_lib_3.sh

The above representation could easily exist in a large program or tool-chain you have set up with a scattered assortment of runnable scripts and libraries, especially when sharing (sourcing/importing) code between your various bash scripts, no matter where they lie.

Imagine you have these constraints/requirements:

1. The entire directory structure above is all stored inside a single GitHub repo.
2. You need anyone to be able to git clone this repo and just have all scripts and imports and things magically work for everybody!
   1. This means you need to use relative imports when importing your bash scripts into each other.
3. You are going to run my_script.sh, for example.
4. You must be able to call my_script.sh from anywhere, meaning: you should be able to be cded into any directory in your entire file system at the time you call this script to run.
5. my_script.sh must use relative imports to import my_lib_1.sh, my_lib_2.sh, and my_lib_3.sh.

Here's how! Basically, we are going to find the path to the script being run, and then use that path as a relative starting point to the other scripts around this script!

Read my full answer on this for more details: How to obtain the full file path, full directory, and base filename of any script being run OR sourced...even when the called script is called from within another bash function or script, or when nested sourcing is being used!.

Full example:

my_script.sh:

#!/usr/bin/env bash

# Get the path to the directory this script is in.
FULL_PATH_TO_SCRIPT="$(realpath "${BASH_SOURCE[-1]}")"
SCRIPT_DIRECTORY="$(dirname "$FULL_PATH_TO_SCRIPT")"

# Now, source the 3 Bash libraries of interests, using relative paths to this
# script!
. "$SCRIPT_DIRECTORY/../my_lib_1.sh"
. "$SCRIPT_DIRECTORY/my_lib_2.sh"
. "$SCRIPT_DIRECTORY/dir3/my_lib_3.sh"

# Now you've sourced (imported) all 3 of those scripts!

That's it! Super easy if you know the commands!

Can Python do this? No, not natively at least. Bash is much easier than Python in this regard! Python doesn't use filesystem paths directly when importing. It's much more complicated and convoluted than that. But, I'm working on an import_helper.py Python module to make this type of thing easy in Python too. I'll publish it shortly as well.

Going further

1. See my full and really-useful Bash floating point library in my eRCaGuy_hello_world repo here: floating_point_math.sh.
2. See some alternative notes of mine on Bash library installation and usage in my readme here: https://github.com/ElectricRCAircraftGuy/eRCaGuy_hello_world/tree/master/bash/libraries

See also

1. This content on my personal website here: GabrielStaples.com: How do you write, import, use, and test libraries in Bash?
2. Testing a bash shell script - this answer mentions this assert.sh Bash repo, which looks really useful for more-robust Bash unit testing!
3. Bash and Test-Driven Development
4. Unit testing Bash scripts

_{Note: I migrated this answer from here, where I had accidentally created a duplicate Q&A.}

Tanga

Answer 5

If you are using Bash, another solution may be:

#!/bin/bash

foo () { ... }
bar () { ... }

[[ "${FUNCNAME[0]}" == "source" ]] && return
code

Teetotalism

Interesting idea, but some explanation of what that does would be helpful. It looks like this is looking for the source built-in at the top of the call stack. However this only seemed to work for me if I source the script with the '.' command rather than typing out "source scriptname" which seemed odd. – Clack

FYI: FUNCNAME can't handle nested scripts, where one script calls or sources another. See my notes and an alternative approach in my answers here and here. – Tanga

Answer 6

I devised this. Let's say our shell library file is the following file, named aLib.sh:

funcs=("a" "b" "c")                   # File's functions' names
for((i=0;i<${#funcs[@]};i++));        # Avoid function collision with existing
do
        declare -f "${funcs[$i]}" >/dev/null
        [ $? -eq 0 ] && echo "!!ATTENTION!! ${funcs[$i]} is already sourced"
done

function a(){
        echo function a
}
function b(){
        echo function b
}
function c(){
        echo function c
}


if [ "$1" == "--source-specific" ];      # Source only specific given as arg
then    
        for((i=0;i<${#funcs[@]};i++));
        do      
                for((j=2;j<=$#;j++));
                do      
                        anArg=$(eval 'echo ${'$j'}')
                        test "${funcs[$i]}" == "$anArg" && continue 2
                done    
                        unset ${funcs[$i]}
        done
fi
unset i j funcs

At the beginning it checks and warns for any function name collision detected. At the end, bash has already sourced all functions, so it frees memory from them and keeps only the ones selected.

Can be used like this:

 user@pc:~$ source aLib.sh --source-specific a c
 user@pc:~$ a; b; c
 function a
 bash: b: command not found
 function c

~

Transcendental