Embedding commands in strings

Question

Consider the following:

#!/bin/tcsh

set thing = 'marker:echo "quoted argument"'
set a = `echo "$thing" | sed 's/\([^:]*\):\(.*\)/\1/'`
set b = `echo "$thing" | sed 's/\([^:]*\):\(.*\)/\2/'`
echo $a
echo $b
$b
echo "quoted argument"

This gives

marker
echo "quoted argument"
"quoted argument"
quoted argument

If $b is echo "quoted argument", why does evaluating $b give a different result from echo "quoted argument"?

---

Since I know tcsh is awful (but it's what I have to use for work), here is the same problem in Bash:

thing='marker:echo "quoted argument"'
a=`echo "$thing" | sed 's/\(.*\):\([^:]*\)/\1/'`
b=`echo "$thing" | sed 's/\(.*\):\([^:]*\)/\2/'`
echo $a
echo $b
$b
echo "quoted argument"

The output is the same. Note that, were I doing this in Bash, I would certainly use a map. I do not have that luxury :). The solution must work in tcsh.

Desired Output

I would like $b to behave just as if I typed the command in myself as I see it:

marker
echo "quoted argument"
quoted argument
quoted argument

_{This is a follow-up question to Accessing array elements with spaces in TCSH.}

Suzan

Consider updating your question to show your required output. Now we're just guessing. Shells are always looking to strip away quoting on the commands/text that it processes. (not really, it just seems like it). Your tcsh output looks correct to me. If you're looking for your final result to include dbl-quotes, i.e. "quoted argument", then you have to escape them so the shell does not consume them OR put extra quoting values in your initial string to begin with. ' ....'\"'....' might work, or you might need up to 5 '\\' chars to protect your dbl-qt from being stripped by shell procing. – Baskin

@Baskin Sorry about that – I'd been working with the problem so long that the goal seemed obvious to me from my question :) Fixed. – Suzan

ah, so you want the result of echo "quoted argument" to appear. While considered evil by many, considering your constrained use, try set thing = 'marker:eval echo "quoted argument"' . (Using eval). I can't test this in my environment. Good luck. – Baskin

@Baskin Thanks for the tip :) I'll try doing this and let you know how I get on; hopefully that can be transformed into an answer. – Suzan

@Baskin It does work :) I moved to accept another answer though, since you seem to be AFK. Many thanks! Interesting information in the answers below, though. – Suzan

Answer 1

Yeah, eval is the "solution" here (well the solution is not to have a command in a string in the first place see http://mywiki.wooledge.org/BashFAQ/050 for more).

The reason you see the quotes when you run $b is because of the order of evaluation of a shell command. The very last thing that the shell does, after all other expansions, is to remote quotes (however it doesn't remove quotes that resulted from any of the expansions).

So when you have b='echo "quoted arguments"' and run $b as the command line what happens is that the variable is expanded so you get echo "quoted arguments" and then that is run as-is.

$ c ()
{
    printf 'argc: %s\n' "$#";
    printf 'argv: %s\n' "$@"
}

$ b='echo "quoted arguments"'

$ c "quoted arguments"
argc: 1
argv: quoted arguments
$ c $b
argc: 3
argv: echo
argv: "quoted
argv: arguments"
$ c "$b"
argc: 1
argv: echo "quoted arguments"
$ eval c $b
argc: 2
argv: echo
argv: quoted arguments
$ eval c "$b"
argc: 2
argv: echo
argv: quoted arguments

Retinite

Ah, so it's similar to brace-expansion then in some macro languages (TeX and m4 come to mind). Since ' has already been 'eaten up' by the shell, as it were, each " is left as is. Correct? – Suzan

As such, calling eval again 'expands' that business once more for the result. Awesome :) And yes, if I were using Bash, I would certainly instruct the user to use function names. Perhaps I can utilize aliases for tcsh, though. Thoughts for any future readers. – Suzan

Yes, eval performs normal shell evaluation of the line and then turns that result into a command and runs that. So eval echo "\"foo\"" first strips the outer quotes and the escapes from the string and ends up with echo foo which is then run. Functions aren't the deal with the solution (c was just to show the expansions) the deal with the real solution is to use arrays which can actually keep arguments separate without needing "extra" quotes. – Retinite

In my use case, I'm already using an array. tcsh doesn't seem to support nested arrays. What I'm effectively implementing is a very limited dictionary structure :) – Suzan

Answer 2

One thing you must keep in mind with command substitution is that each pipe and each command you string together executes within its own subshell. Each time that happens, the shell process the command or string you provide:

If $b is echo "quoted argument", why does evaluating $b give a different result from echo "quoted argument"?

set thing = 'marker:echo "quoted argument"'
set b = `echo "$thing" | sed 's/\([^:]*\):\(.*\)/\2/'`
echo $b
echo "quoted argument"

In the case of b, you are assigning the return from sed exactly as it is returned to b including the quotes. They become part of b. So echo $b is equivalent to echo '"quoted argument"'. Whereas, your echo "quoted argument" prints the string as the characters contained within the quotes, the shell removing the literal quotes.

Sorry for the initial confusion.

Antilog