IFS=: leads to different behavior while looping over colon-separated values

Question

Experiment 1

Here is my first script in file named foo.sh.

IFS=:
for i in foo:bar:baz
do
    echo $i
done

This produces the following output.

$ bash foo.sh
foo bar baz

Experiment 2

This is my second script.

IFS=:
for i in foo:bar:baz
do
    unset IFS
    echo $i
done

This produces the following output.

$ bash foo.sh
foo:bar:baz

Experiment 3

This is my third script.

IFS=:
var=foo:bar:baz
for i in $var
do
    echo $i
done

This produces the following output.

$ bash foo.sh
foo
bar
baz

Why is the output different in all three cases? Can you explain the rules behind the interpretation of IFS and the commands that leads to this different outputs?

Huehuebner

You're conflating two different things: The behavior of the for i in foo:bar:baz, and the behavior of the unquoted expansion in echo $i. If you were testing with, say, echo "$i", then that would let you isolate only the behavior of for i in foo:bar:baz (and how it doesn't change at all based on IFS), which is arguably the core of your question -- whereas by contrast, a question that only asked about behavior of echo $i with no surrounding for would be a simple dupe, which could be answered by a pointer to questions and answers that already exist. – Wellgroomed

Answer 1

I found this a very interesting experiment. Thank you for that.

---

To understand what is going on, the relevant section from man bash is this:

  Word Splitting
      The  shell  scans the results of parameter expansion, command substitu-
      tion, and arithmetic expansion that did not occur within double  quotes
      for word splitting.

The key is the "results of ..." part, and it's very subtle. That is, word splitting happens on the result of certain operations, as listed there: the result of parameter expansion, the result of command substitution, and so on. Word splitting is not performed on string literals such as foo:bar:baz.

Let's see how this logic plays out in the context of your examples.

Experiment 1

IFS=:
for i in foo:bar:baz
do
    echo $i
done

This produces the following output:

foo bar baz

No word splitting is performed on the literal foo:bar:baz, so it doesn't matter what is the value of IFS, as far as the for loop is concerned.

Word splitting is performed after parameter expansion on the value of $i, so foo:bar:baz is split to 3 words, and passed to echo, so the output is foo bar baz.

Experiment 2

IFS=:
for i in foo:bar:baz
do
    unset IFS
    echo $i
done

This produces the following output:

foo:bar:baz

Once again, no word splitting is performed on the literal foo:bar:baz, so it doesn't matter what is the value of IFS, as far as the for loop is concerned.

Word splitting is performed after parameter expansion on the value of $i, but since IFS was unset, its default value is used to perform the split, which is <space><tab><newline>. But since foo:bar:baz doesn't contain any of those, it remains intact, so the output is foo:bar:baz.

Experiment 3

IFS=:
var=foo:bar:baz
for i in $var
do
    echo $i
done

This produces the following output:

foo
bar
baz

After the parameter expansion of $var, word splitting is performed using the value of IFS, and so for has 3 values to iterate over, foo, bar, and baz. The behavior of echo is trivial here, the output is one word per line.

---

The bottomline is: word splitting is not performed on literal values. Word splitting is only performed on the result of certain operations.

This is not all that surprising. A string literal is much like an expression written enclosed in double-quotes, and you wouldn't expect word splitting on "...".

Lohr

Great answer, this should get the bounty! Similar question/answer here as well: Word splitting in Bash with printf and with IFS set to non-whitespace character. – Xenolith