Posix shell: distinguish between empty and not existing variable

Question

In pure /bin/sh how can I distinguish between an empty variable, an unset variable and a not existing (not defined) variable.

Here are the case:

# Case 1: not existing
echo "${foo}"

# Case 2: unset
foo=
echo "${foo}"

# Case 3: Empty
foo=""
echo "${foo}"

Now I would like to check for each of those three cases. If case 2 and case 3 are actually the same, then I must at least be able to distinguish between them and case 1.

Any idea?

UPDATE Solved thanks to Matteo

Here is how the code looks like:

#foo  <-- not defined
bar1=
bar2=""
bar3="a"

if ! set | grep '^foo=' >/dev/null 2>&1; then
    echo "foo does not exist"
elif [ -z "${foo}" ]; then
    echo "foo is empty"
else
    echo "foo has a value"
fi

if ! set | grep '^bar1=' >/dev/null 2>&1; then
    echo "bar1 does not exist"
elif [ -z "${bar1}" ]; then
    echo "bar1 is empty"
else
    echo "bar1 has a value"
fi

if ! set | grep '^bar2=' >/dev/null 2>&1; then
    echo "bar2 does not exist"
elif [ -z "${bar2}" ]; then
    echo "bar2 is empty"
else
    echo "bar2 has a value"
fi


if ! set | grep '^bar3=' >/dev/null 2>&1; then
    echo "bar3 does not exist"
elif [ -z "${bar3}" ]; then
    echo "bar3 is empty"
else
    echo "bar3 has a value"
fi

And the results:

foo does not exist
bar1 is empty
bar2 is empty
bar3 has a value

Monotonous

Your case 2 and case 3 are identical. foo= defines foo to be the empty string exactly the same way that foo="" does. – Dilettantism

Answer 1

You can use set

If no options or arguments are specified, set shall write the names and values of all shell variables in the collation sequence of the current locale. Each name shall start on a separate line, using the format:

You can list all the variables (set) and grep for the variable name you want to check

set | grep '^foo='

Nimmons

function foo=bar { :; } will fool this in bash, and bar=$'\nfoo=nope' will fool it in most other shells. – Welladvised

It won't if your shell is POSIX compliant. If you're using /bin/bash and not /bin/sh, you need to set -o posix and it works with caveats. – Melody

Answer 2

I dont know about sh, but in bash and dash you can do echo ${TEST:?Error} for case 1 vs. case 2/3. And from quick glance at wikibooks, it seems like it should work for bourne shell too.

You can use it like this in bash and dash (use $? to get the error code)

echo ${TEST:?"Error"}
bash: TEST: Error
[lf@dell:~/tmp/soTest] echo $?
1
[lf@dell:~/tmp/soTest] TEST2="ok"
[lf@dell:~/tmp/soTest] echo ${TEST2:?"Error"}
ok
[lf@dell:~/tmp/soTest] echo $?
0
[lf@dell:~/tmp/soTest] dash
$ echo ${TEST3:?"Error"}       
dash: 1: TEST3: Error
$ TEST3=ok
$ echo ${TEST3:?"Error"}
ok

Abbotsen

The syntax is POSIX but it will not distinguish between a non existing variable and an empty variable – Nimmons

@Nimmons I thought of using $? to script the cases, see my edit. – Abbotsen

${TEST:?Error} does not distinguish between undefined and empty. ${TEST?Error} does. – Dilettantism

This will cause a non-interactive shell/script to exit immediately if the variable is unset, you will not be able to check the value of $? as you are doing here. – Huddle

Answer 3

You can use ${var?} syntax to throw an error if var is unset and ${var:?} to throw an error if var is unset or empty. For a concrete example:

$ unset foo
$ test -z "${foo?unset}" && echo foo is empty || echo foo is set to $foo
-bash: foo: unset
$ foo=
$ test -z "${foo?unset}" && echo foo is empty || echo foo is set to $foo
foo is empty
$ foo=bar
$ test -z "${foo?unset}" && echo foo is empty || echo foo is set to $foo
foo is set to bar

For the non-interactive shell, you can use ${+}. For example:

#!/bin/sh

check() {
        case ${foo:+set} in
        set) echo foo is set to "'$foo'";;
        *)
                case ${foo+empty} in
                empty) echo foo is empty;;
                *) echo foo is unset;;
                esac
        esac
}

unset foo
check
foo=
check
foo=hello
check

Dilettantism

This will cause a non-interactive shell/script to exit immediately upon evaluation of "${foo?unset}" if foo is unset. – Huddle

@Huddle A valid concern. I've added code to be used in the non-interactive case. – Dilettantism

Answer 4

I don't see a correct answer here yet that is POSIX compliant. First let me reiterate William Pursell's assertion that the code foo= is indeed setting the variable foo to an empty value the same as foo="". For foo to be unset, it must either never be set or unset with unset foo.

Matteo's answer is correct, but there are caveats. When you run set in bash and posix mode is disabled, it also prints all of the defined functions as well. This can be suppressed like this:

isvar() (
    [ -n "$BASH" ] && set -o posix
    set | grep -q "^$1="
)

By writing it as a sub-shell function, we don't need to worry about what the state of bash's posix setting after we're done.

However, you can still get false-positives from variables whose values contain carriage returns, so understand that this is not 100% foolproof.

$ a="
> b=2
> "
$ set | grep ^b=
b=2

So for maximum correctness you can exploit bash's -v test, when available.

isvar() {
    if [ -n "$BASH" ]; then
        [ -v "$1" ]
    else
        set | grep -q "^$1="
    fi
}

Perhaps somebody has a library somewhere that supports other shells' extensions as well. Essentially, this is a weakness in the POSIX specification and it just hasn't been seen as warranting amendment.

Melody

Answer 5

if ! (: "${foo?}") >/dev/null 2>&1; then
    # foo is unset
elif [ "X$foo" '=' 'X' ]; then
    # foo is null/empty
else
    # foo is not null/empty
fi

Huddle

Remember that Stack Overflow isn't just intended to solve the immediate problem, but also to help future readers find solutions to similar problems, which requires understanding the underlying code. This is especially important for members of our community who are beginners, and not familiar with the syntax. Given that, can you edit your answer to include an explanation of what you're doing and why you believe it is the best approach? – Brabazon

Answer 6

POSIX shell actually knows three variable states:

1. Variable is set to a value.
2. Variable is set but Null.
3. Variable is not set at all (unset).

The most common check used is -z but this test actually checks for an empty string after the variable has been expanded. Both, an unset and Null variable expand to an empty string:

null=
value="x"
[ -z "$unset" ] && echo '$unset is empty'
[ -z "$empty" ] && echo '$empty is empty'
[ -z "$value" ] && echo '$value is empty'

prints

$unset is empty
$empty is empty

https://onlinegdb.com/gH0L4YywZ

There is no way to directly test a variable being unset but there is Parameter Expansion.

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_02

E.g. to check if a variable is set at all, use +

null=
value="x"
[ -n "${unset+x}" ] && echo '$unset is set'
[ -n "${null+x}" ] && echo '$null is set'
[ -n "${value+x}" ] && echo '$value is set'

prints

$null is set
$value is set

which is the expected result. https://onlinegdb.com/oPC3i65_We

Note that instead of +x you could use anything, also +IS_SET if you like. +... will result in an empty string if unset and in a string containing ... in any other case, no matter what ... is and no matter what non-empty value a variable might have. Thus it will also never conflict with a real value, so value also being x here is irrelevant.

Also note that there is no such thing as an empty string in POSIX. An empty string is the same as being Null. POSIX variables are always strings, they are only interpreted as integers if you explicitly request that but results of math operations are stored as strings again. That's why value="" and value= are effectively the same instruction. They create a string with zero characters and that's a Null value in POSIX, as characters are what makes the variable's content and a variable having zero characters has no content, thus it is Null.

Anette