Loop over space-separated string in zsh

Question

What controls the environment to know to split by space in zsh?

I'm sure it's something simple but in all my searching have yet to figure it out what controls it.

Trying to loop over items in a space-separated string like so:

s='foo bar baz'
for i in $s; do
  echo "$i END"
done
# foo bar baz END

# ---

s='foo bar baz'
a=( $s )
echo ${a[0]} # (empty)
echo ${a[1]} # foo bar baz

# ---

s='foo bar baz'
IFS=' ' read a <<< $s
for i in "${a[@]}"; do
  echo "$i END"
done
# foo bar baz END

The different methods work via sh and bash, but in a shell with oh-my-zsh I'm unable to separate by space, getting the results above. May not be oh-my-zsh - but looking to understand what drives this.

Working example from bash:

s='foo bar baz'
for i in $s; do
  echo "$i END"
done
# foo END
# bar END
# baz END

Efferent

Are you asking about zsh word splitting behaviour, and how/why it is different from the one on Bash / POSIX sh? – Modulate

oh-my-zsh is just a framework for customizing the zsh shell; if you're using oh-my-zsh, then you're using zsh (not bash). zsh is significantly different from most other shells, so you need to look for documentation about zsh, not bash or any other shell (and not so much oh-my-zsh, because again, it's just a way of customizing zsh). – Stacistacia

@BenjaminW. - yes and how to get zsh to split by word. – Efferent

Answer 1

Zsh and bash are two different programming languages. They're similar, but not identical. In bash, and more generally in Bourne-style shells (sh, dash, ksh, …), an unquoted variable expansion $foo does the following:

1. Take the value of the variable foo, which is a string. (If there is no variable foo, take the empty string.)
2. Split the string into whitespace-separated parts. (More generally, the value of the IFS variable determines how the string is split; I won't go into all the details here.) The result is a list of strings.
3. For every element in the list, if it is a globbing pattern, i.e. if it contains at least one wildcard character *?\[ (and possibly more depending on some shell options), and that pattern matches at least one file name, then the element is replaced by the list of matching file names. Elements that don't contain any wildcard character, and elements that contain a wildcard character but don't match any file name, are left alone. The result is again a list of strings.

Zsh is mostly a Bourne-style shell, but it has some differences, and this is the main one: $foo has the following, simpler behavior.

1. Take the value of the variable foo, which is a string. (If there is no variable foo, take the empty string.)
2. If this results in an empty word, this word is eliminated. (So for example $foo$bar is only eliminated if both foo and bar are empty or unset.)

Note that in sh or bash, $foo only works to split a string if it doesn't contain any wildcard character or if globbing is disabled with set -f.

To split a string at whitespace in zsh, there are two simple methods:

• Use the = parameter expansion specified to apply IFS word splitting. For example $=foo splits at whitespace as determined by IFS.
• Use the p parameter expansion flag. For example ${(p: :)foo} splits at spaces (not tabs or newlines).

This has nothing to do with oh-my-zsh, which is a plugin to configure zsh for interactive use.

Satanic

You can also enable word splitting of unquoted expansions with setopt sh_word_split in individual scripts. – Banquet

@Banquet Yes, but this affects every single parameter expansion. There's hardly ever any reason to explicitly enable this option: if you need bash/sh/ksh compatibility, use emulate. – Bankroll

@Gilles'SO-stopbeingevil' I figured it was nothing to do with oh-my-zsh, but possibly zsh - mentioned OMZ incase it had opinionated settings but probably should have mentioned separately. Even with this explanation, setting the IFS before the FOR - IN loops does nothing and parameter flag expansion (foo='bar baz'; ${(p: :)foo};) gives error 'zsh: error in flags'. Setting the setopt sh_word_split works however. I'm going to research what this sets and why it's different by default. – Efferent

You both lead me right to what I was looking for - the default of zsh does not do word splitting of unquoted parameter expansions - https://mcmap.net/q/448430/-variable-expansion-is-different-in-zsh-from-that-in-bash. Don't know how I missed that when skimming the manual. – Efferent

While this answers the question you asked, what you almost certainly want to actually do instead of these suggestions is to use arrays. That would apply for a bash or ksh script too. Only in sh where there are no arrays is automatic word splitting a desirable feature but it makes it far harder to not have scripts break where e.g. filenames contain spaces. – Plat

@Plat agreed, but it's consuming stdout from a third party solution that spits out the values I need as space-separated IDs, sadly. Just making sure the script runs both locally and within the CI job it's built for. – Efferent

Answer 2

Just ${(p: :)foo} didn't work for me, and was giving a a zsh: error in flags error. After reading Parameter-Expansion doc, I see that it should be ${(ps: :)foo}. Even the flag p explanation in the doc uses the additional s flag.

The p flag doc says:

p  :  Recognize the same escape sequences as the print builtin in string arguments to any of the flags described below that follow this argument.

So what I ended up using was just ${(s: :)foo}. See the example for the behavior where only space is used as the separator, contiguous spaces are treated as one, while tabs and newlines are preserved as is:

> FRUITS="apple\tbanana grapes orange      passion_fruit\nwatermelon"

> for F in ${(ps: :)FRUITS}; do echo "GOT: <$F>"; done
GOT: <apple banana>
GOT: <grapes>
GOT: <orange>
GOT: <passion_fruit
watermelon>

Cysticercus