Repeated negation (!) operators in bash do not negate each other

Question

So this is more an oddity I've come up against than something I really want to use. But I found something I didn't understand with the bash extended test syntax.

Check this out (included my shell version in case it matters):

34>$SHELL --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16)
Copyright (C) 2007 Free Software Foundation, Inc.
35>[ ! -d /tmp ] && echo Hi
36>[ ! ! -d /tmp ] && echo Hi
Hi
37>[[ ! -d /tmp ]] && echo Hi
38>[[ ! ! -d /tmp ]] && echo Hi
39>

OK, so lines 35 and 36, using the normal test, operate as I expect. The single bang doesn't print a line (because /tmp exists), and the double bang does.

Line 37, using extended bash syntax, also doesn't print anything, as I would expect. But line 38 doesn't either! This is surprising to me; it indicates that the directory doesn't exist, but also doesn't not exist?

Searching for information on this has been frustrating. Am I missing something here? An unmentioned syntax error? I just want to understand why this happens.

Hypoderm

If you want this to be well-defined, move the !s outside. ! ! [[ -d /tmp ]] relies only on behavior that's actually part of the test suite for the shell, whereas the historical understanding of test behavior referenced in answer E1 in the FAQ in bash's codebase does not require multiple !s inside a single test to be valid (or to have any other specific/particular behavior, making the result at hand here as legitimate as any other -- ain't undefined behavior fun?). – Kodok

I'm reading E1 as describing why ! doesn't bind as tightly as -o, nothing more. [[ ! ! -d /tmp ]] failing just looks like a bug to me; [[ ! (! -d /tmp) ]] works correctly. – Numbat

To clarify, I don't WANT to ever actually use this code. I was just curious to understand why it behaved like it does. :) – Hypoderm

I can reproduce this issue in bash 4.4.18(1)-release. Interestingly, ksh does the same thing. – Ronnaronnholm

Answer 1

Due to the use of a flag in the BASH code which is not toggled in this particular case, only the first instance of ! matters, unless brackets are separating them.

To find this out, first I took a look at the Bash(1) Man Page:

Reserved Words

Reserved words are words that have a special meaning to the shell. [...] words are recognized as reserved when unquoted and either the first word of a simple command (see SHELL GRAMMAR below) or the third word of a case or for command

Which hints to the usage of the ! reserved word.

However, this doesn't really explain it very well. So I took a look at the source code (version 4.4.18) for Bash.

It looks like command.h contains a flag which is set for a command, when it detects the ! symbol:

#define CMD_INVERT_RETURN  0x04 /* Invert the exit value. */

This is used several times in the execute_cmd.c file:

invert = (command->flags & CMD_INVERT_RETURN) != 0;

But this file seems to just check for the presence of the flag. I believe the parsing of it is done in another file.

On line 4507 of parse.y, we can see that it seems to just be set, not toggled. This means that it doesn't matter how many occurrences of BANG (!) there is, it'll only set the flag once.

else if (tok == BANG || (tok == WORD && (yylval.word->word[0] == '!' && yylval.word->word[1] == '\0'))) {
  if (tok == WORD)
dispose_word (yylval.word); /* not needed */
  term = cond_term ();
  if (term)
term->flags |= CMD_INVERT_RETURN;
}

I find this behaviour strange, since later in the code, there is support for toggling this value, on line 1201 of parse.y, which relates to pipelines (formatted for readability)

pipeline_command: pipeline
{ $$ = $1; }            
    | BANG pipeline_command {
        if ($2)
            $2->flags ^= CMD_INVERT_RETURN; /* toggle */
        $$ = $2;
    }

Autarch

That certainly matches what the behavior is, but it seems weird. I don't think I'd ever expect any programming language to make ! ! true return false, and to not mention it in extended test syntax documentation, if it is a deliberate choice, seems unhelpful – Hypoderm