A confusion about ${array[*]} versus ${array[@]} in the context of a bash completion

Question

I'm taking a stab at writing a bash completion for the first time, and I'm a bit confused about about the two ways of dereferencing bash arrays (${array[@]} and ${array[*]}).

Here's the relevant chunk of code (it works, but I would like to understand it better):

_switch()
{
    local cur perls
    local ROOT=${PERLBREW_ROOT:-$HOME/perl5/perlbrew}
    COMPREPLY=()
    cur=${COMP_WORDS[COMP_CWORD]}
    perls=($ROOT/perls/perl-*)
    # remove all but the final part of the name
    perls=(${perls[*]##*/})

    COMPREPLY=( $( compgen -W "${perls[*]} /usr/bin/perl" -- ${cur} ) )
}

bash's documentation says:

Any element of an array may be referenced using ${name[subscript]}. The braces are required to avoid conflicts with the shell's filename expansion operators. If the subscript is ‘@’ or ‘*’, the word expands to all members of the array name. These subscripts differ only when the word appears within double quotes. If the word is double-quoted, ${name[*]} expands to a single word with the value of each array member separated by the first character of the IFS variable, and ${name[@]} expands each element of name to a separate word.

Now I think I understand that compgen -W expects a string containing a wordlist of possible alternatives, but in this context I don't understand what "${name[@]} expands each element of name to a separate word" means.

Long story short: ${array[*]} works; ${array[@]} doesn't. I would like to know why, and I would like to understand better what exactly ${array[@]} expands into.

Eadmund

Answer 1

(This is an expansion of my comment on Kaleb Pederson's answer -- see that answer for a more general treatment of [@] vs [*].)

When bash (or any similar shell) parses a command line, it splits it into a series of "words" (which I will call "shell-words" to avoid confusion later). Generally, shell-words are separated by spaces (or other whitespace), but spaces can be included in a shell-word by escaping or quoting them. The difference between [@] and [*]-expanded arrays in double-quotes is that "${myarray[@]}" leads to each element of the array being treated as a separate shell-word, while "${myarray[*]}" results in a single shell-word with all of the elements of the array separated by spaces (or whatever the first character of IFS is).

Usually, the [@] behavior is what you want. Suppose we have perls=(perl-one perl-two) and use ls "${perls[*]}" -- that's equivalent to ls "perl-one perl-two", which will look for single file named perl-one perl-two, which is probably not what you wanted. ls "${perls[@]}" is equivalent to ls "perl-one" "perl-two", which is much more likely to do something useful.

Providing a list of completion words (which I will call comp-words to avoid confusion with shell-words) to compgen is different; the -W option takes a list of comp-words, but it must be in the form of a single shell-word with the comp-words separated by spaces. Note that command options that take arguments always (at least as far as I know) take a single shell-word -- otherwise there'd be no way to tell when the arguments to the option end, and the regular command arguments (/other option flags) begin.

In more detail:

perls=(perl-one perl-two)
compgen -W "${perls[*]} /usr/bin/perl" -- ${cur}

is equivalent to:

compgen -W "perl-one perl-two /usr/bin/perl" -- ${cur}

...which does what you want. On the other hand,

perls=(perl-one perl-two)
compgen -W "${perls[@]} /usr/bin/perl" -- ${cur}

is equivalent to:

compgen -W "perl-one" "perl-two /usr/bin/perl" -- ${cur}

...which is complete nonsense: "perl-one" is the only comp-word attached to the -W flag, and the first real argument -- which compgen will take as the string to be completed -- is "perl-two /usr/bin/perl". I'd expect compgen to complain that it's been given extra arguments ("--" and whatever's in $cur), but apparently it just ignores them.

Shatterproof

This is excellent; thanks. I really wish it blew up more loudly, but this at least clarifies why it didn't work. – Eadmund

Answer 2

Your title asks about ${array[@]} versus ${array[*]} (both within {}) but then you ask about $array[*] versus $array[@] (both without {}) which is a bit confusing. I'll answer both (within {}):

When you quote an array variable and use @ as a subscript, each element of the array is expanded to its full content regardless of whitespace (actually, one of $IFS) that may be present within that content. When you use the asterisk (*) as the subscript (regardless of whether it's quoted or not) it may expand to new content created by breaking up each array element's content at $IFS.

Here's the example script:

#!/bin/sh

myarray[0]="one"
myarray[1]="two"
myarray[3]="three four"

echo "with quotes around myarray[*]"
for x in "${myarray[*]}"; do
        echo "ARG[*]: '$x'"
done

echo "with quotes around myarray[@]"
for x in "${myarray[@]}"; do
        echo "ARG[@]: '$x'"
done

echo "without quotes around myarray[*]"
for x in ${myarray[*]}; do
        echo "ARG[*]: '$x'"
done

echo "without quotes around myarray[@]"
for x in ${myarray[@]}; do
        echo "ARG[@]: '$x'"
done

And here's it's output:

with quotes around myarray[*]
ARG[*]: 'one two three four'
with quotes around myarray[@]
ARG[@]: 'one'
ARG[@]: 'two'
ARG[@]: 'three four'
without quotes around myarray[*]
ARG[*]: 'one'
ARG[*]: 'two'
ARG[*]: 'three'
ARG[*]: 'four'
without quotes around myarray[@]
ARG[@]: 'one'
ARG[@]: 'two'
ARG[@]: 'three'
ARG[@]: 'four'

I personally usually want "${myarray[@]}". Now, to answer the second part of your question, ${array[@]} versus $array[@].

Quoting the bash docs, which you quoted:

The braces are required to avoid conflicts with the shell's filename expansion operators.

$ myarray=
$ myarray[0]="one"
$ myarray[1]="two"
$ echo ${myarray[@]}
one two

But, when you do $myarray[@], the dollar sign is tightly bound to myarray so it is evaluated before the [@]. For example:

$ ls $myarray[@]
ls: cannot access one[@]: No such file or directory

But, as noted in the documentation, the brackets are for filename expansion, so let's try this:

$ touch one@
$ ls $myarray[@]
one@

Now we can see that the filename expansion happened after the $myarray exapansion.

And one more note, $myarray without a subscript expands to the first value of the array:

$ myarray[0]="one four"
$ echo $myarray[5]
one four[5]

Buckler

Also see this regarding how IFS affects the output differently depending on @ vs. * and quoted vs. unquoted. – Archambault

I apologize, since it's pretty important in this context, but I always meant ${array[*]} or ${array[@]}. The lack of braces was simply carelessness. Beyond that, can you explain what ${array[*]} would expand into in the compgen command? That is, in that context what does it mean to expand the array into each of its elements separately? – Eadmund

To put this another way, you (like almost every source) say that ${array[@]} is usually the way to go. What I'm trying to understand is why in this case only ${array[*]} works. – Eadmund

It's because the wordlist supplied with the -W option must be given as a single word (which compgen then splits based on IFS). If it's split into separate words before being handed to compgen (which is what [@] does), compgen will think that only the first one goes with -W, and the rest are regular arguments (and I think it only expects one argument, and will therefore barf). – Shatterproof

@Gordon: Move that to an answer, and I'll accept it. That's what I really wanted to know. Thanks. (Btw, it doesn't barf in an obvious way. It silently barfs - which makes it hard to know what went wrong.) – Eadmund

@Eadmund - Looks like you have your answer. I'll give @Gordon a chance to add an answer or will otherwise edit my post giving him credit. – Buckler

Just curious, where is the myarray[2]? - was not declared by some explicit reason? – Blandish

unix.stackexchange.com/a/753925/527038 – Kendrickkendricks