Tricky brace expansion in shell

Question

When using a POSIX shell, the following

touch {quick,man,strong}ly

expands to

touch quickly manly strongly

Which will touch the files quickly, manly, and strongly, but is it possible to dynamically create the expansion? For example, the following illustrates what I want to do, but does not work because of the order of expansion:

TEST=quick,man,strong    #possibly output from a program
echo {$TEST}ly

Is there any way to achieve this? I do not mind constricting myself to Bash if need be. I would also like to avoid loops. The expansion should be given as complete arguments to any arbitrary program (i.e. the program cannot be called once for each file, it can only be called once for all files). I know about xargs but I'm hoping it can all be done from the shell somehow.

Teddi

Brace expansion is not defined in the POSIX shell: pubs.opengroup.org/onlinepubs/9699919799/utilities/… – Gurgitation

@pabouk: That's correct, now I know brace expansion isn't POSIX, but 7 years ago (when I asked this question) I didn't know that :) – Teddi

Answer 1

... There is so much wrong with using eval. What you're asking is only possible with eval, BUT what you might want is easily possible without having to resort to bash bug-central.

Use arrays! Whenever you need to keep multiple items in one datatype, you need (or, should use) an array.

TEST=(quick man strong)
touch "${TEST[@]/%/ly}"

That does exactly what you want without the thousand bugs and security issues introduced and concealed in the other suggestions here.

The way it works is:

• "${foo[@]}": Expands the array named foo by expanding each of its elements, properly quoted. Don't forget the quotes!
• ${foo/a/b}: This is a type of parameter expansion that replaces the first a in foo's expansion by a b. In this type of expansion you can use % to signify the end of the expanded value, sort of like $ in regular expressions.
• Put all that together and "${foo[@]/%/ly}" will expand each element of foo, properly quote it as a separate argument, and replace each element's end by ly.

Disburse

Any suggestions for that for multiple levels of expansion? for instance, echo {a,b}/{c,d}/{e,f} gives a/c/e a/c/f a/d/e a/d/f b/c/e b/c/f b/d/e b/d/f ; could I achieve that with arrays, without having to write loops? – Hereunder

@Hereunder You could assign the brace expansion to an array. files=( {a,b}/{c,d}/{e,f} ); touch "${files[@]}" – Disburse

Notice you may use # instead of % to prefix items: ${TEST[@]/#/ly} gives lyquick lyman lystrong – Malka

Answer 2

In bash, you can do this:

#!/bin/bash
TEST=quick,man,strong
eval echo $(echo {$TEST}ly)
#eval touch $(echo {$TEST}ly)

That last line is commented out but will touch the specified files.

Jonas

Just so you know, there are masses of issues with that solution, which just hide because you're using only just letters and commas in your TEST parameter. Any character that has a special meaning in bash will screw it up, especially so because you even completely failed to quote anything. I realize it's a demo with the given data but using this solution in real world applications is a sin. – Disburse

@paxdiablo: You can use eval echo {$TEST}ly instead of eval echo $(echo {$TEST}ly). AFAIK the extra echo doesn't add anything. – Daven

@lhunath: you seem to despise eval. Is there something particularly wrong with this that will cause a problem, or do you simply consider anything with eval to be automatically evil? – Daven

@iconoclast: The golden rule of handling data: "Don't merge two strings of different context." If you violate that rule, you equate the context of one string to the context of the latter, which is always a bug and usually results in arbitrary code execution. In this case, you're merging data such as "quick" and "man" and "strong" with bash code, such as "echo" and brace expansion. The result is that you're pretending that your data is bash code, which it isn't. When 'strong' becomes 'strong}; rm -rf ~; #' you're in trouble. – Disburse

@lhunath: why are you assuming that the value of $TEST is coming from an untrusted source? Also, by "bug" you seem to mean not an actual flaw (where something goes unexpectedly) but a risk--a hypothetical opportunity for something to go wrong. I don't think most people would use the word as you seem to be using it. (Although I assume we all agree that using eval involves risks, and applying eval to data from an untrusted source is practically suicidal.) – Daven

@iconoclast: A bug is something that happens which is not what one expects. When a string of bytes means one thing to you but gets interpreted as though it means something else, then that latter interpretation is a bug: It does not do what you expect. In this scenario, the TEST string gets interpreted as bash code. Which is not what is expected; the expected meaning of it is a series of comma-delimited elements. The bug only shows itself when you have an element that contains certain characters; but before then, the bug is still there, just hidden in the code. Waiting to cause havoc. – Disburse

@lhunath: so you consider all uses of eval to be bugs, then. – Daven

@iconoclast: Almost all, yes. There are almost always preferable ways of doing something over eval. If eval ends up to be the only method; you're probably using the wrong language for the task anyway. A few rare "acceptable" cases are eval'ing in variables generated as output of a program intended for injection into the program's parent shell (like ssh-agent, etc. tend to do). – Disburse

Answer 3

Zsh can easily do that:

TEST=quick,man,strong
print ${(s:,:)^TEST}ly

Variable content is splitted at commas, then each element is distributed to the string around the braces:

quickly manly strongly

Jenaejenda

Answer 4

Taking inspiration from the answers above:

$ TEST=quick,man,strong
$ touch $(eval echo {$TEST}ly)

Saccharometer