In Bash, is there a way to expand variables twice in double quotes?

Question

For debugging my scripts, I would like to add the internal variables $FUNCNAME and $LINENO at the beginning of each of my outputs, so I know what function and line number the output occurs on.

foo(){
    local bar="something"
    echo "$FUNCNAME $LINENO: I just set bar to $bar"
}

But since there will be many debugging outputs, it would be cleaner if I could do something like the following:

foo(){
    local trace='$FUNCNAME $LINENO'
    local bar="something"
    echo "$trace: I just set bar to $bar"
}

But the above literally outputs: "$FUNCNAME $LINENO: I just set bar to something" I think it does this because double quotes only expands variables inside once.

Is there a syntactically clean way to expand variables twice in the same line?

Paracasein

Oof. It'd be easy if you only had one variable name in your variable -- in that case it's just standard indirect expansion. By contrast, if you want to expand a template with an arbitrary number of variables, you're getting into eval space, which is distinctly not safe at all. – Tical

this almost works - if local trace=FUNCNAME then you could write echo ${!trace} ": I just set bar to $bar" – Croatia

@codeforester ...that's a pretty heavy-handed title edit. Completely removes context from the one-line summary at the top of my (original) answer, f/e, and lends itself to answers that don't address the OP's direct/immediate question about expansion at all. – Tical

I thought so too... Now that you are telling me, I just rolled it back. – Keelia

Answer 1

You cannot safely evaluate expansions twice when handling runtime data.

There are means to do re-evaluation, but they require trusting your data -- in the NSA system design sense of the word: "A trusted component is one that can break your system when it fails".

See BashFAQ #48 for a detailed discussion. Keep in mind that if you could be logging filenames, that any character except NUL can be present in a UNIX filename. $(rm -rf ~)'$(rm -rf ~)'.txt is a legal name. * is a legal name.

Consider a different approach:

#!/usr/bin/env bash

trace() { echo "${FUNCNAME[1]}:${BASH_LINENO[0]}: $*" >&2; }

foo() {
        bar=baz
        trace "I just set bar to $bar"
}

foo

...which, when run with bash 4.4.19(1)-release, emits:

foo:7: I just set bar to baz

---

Note the use of ${BASH_LINENO[0]} and ${FUNCNAME[1]}; this is because BASH_LINENO is defined as follows:

An array variable whose members are the line numbers in source files where each corresponding member of FUNCNAME was invoked.

Thus, FUNCNAME[0] is trace, whereas FUNCNAME[1] is foo; whereas BASH_LINENO[0] is the line from which trace was called -- a line which is inside the function foo.

Tical

Answer 2

Yes to double expansion; but no, it won't do what you are hoping for.

Yes, bash offers a way to do "double expansion" of a variable, aka, a way to first interpret a variable, then take that as the name of some other variable, where the other variable is what's to actually be expanded. This is called "indirection". With "indirection", bash allows a shell variable to reference another shell variable, with the final value coming from the referenced variable. So, a bash variable can be passed by reference.

The syntax is just the normal braces style expansion, but with an exclamation mark prepended to the name.

${!VARNAME}

It is used like this:

BAR="my final value";
FOO=BAR
echo ${!FOO};

...which produces this output...

my final value

No, you can't use this mechanism to do the same as $( eval "echo $VAR1 $VAR2" ). The result of the first interpretation must be exactly the name of a shell variable. It does not accept a string, and does not understand the dollar sign. So this won't work:

BAR="my final value";
FOO='$BAR'; # The dollar sign confuses things
echo ${!FOO}; # Fails because there is no variable named '$BAR'

So, it does not solve your ultimate quest. None-the-less, indirection can be a powerful tool.

Redeploy

We have many preexisting SO questions about how to use indirection -- Dynamic variable names in bash is perhaps the most canonical. BashFAQ #6 is another great resource. All that said, if I had believed the OP here to be asking a question on this subject, I would have closed the question as duplicate rather than answering it. – Tical

Answer 3

Although eval has its dangers, getting a second expansion is what it does:

foo(){
    local trace='$FUNCNAME $LINENO'
    local bar="something"
    eval echo "$trace: I just set bar to $bar"
}

foo

Gives:

foo 6: I just set bar to something

Just be careful not to eval anything that has come from external sources, since you could get a command injected into the string.

Gelt

Not just command injection risks -- you also can't trust the log to be accurate (and thus, to be useful for debugging). If you set bar='*' instead of bar=something, the quoting won't survive through to echo, so you'd get a list of filenames in the log. – Tical

eval 'echo '"$trace"'": I just set bar to $bar"' would be safer (only expanding $trace, and not the rest of the string, twice), though obviously there's an ease-of-use hit. – Tical

@CharlesDuffy: accepted. This use-case seemed what eval was originally designed for and I felt that someone should show it as a solution. – Gelt

far worse than "*" is ";rm -irf /" – Misdeal

I suggest that if someone could hack things to inject that command then you have bigger problems. – Gelt

@cdarke, ...in a log function? The point of trace-level logging is (often) to show what data you're working with at runtime, to allow consideration of how a program is behaving in real-life circumstances. That data very, very often comes from somewhere (uploaded filenames, file content created by other tools, logs written by other processes who got the strings they substituted into those logs from who-knows-where, &c) less trusted than your code. – Tical