How can I store a command in a variable in a shell script?

Question

I would like to store a command to use at a later time in a variable (not the output of the command, but the command itself).

I have a simple script as follows:

command="ls";
echo "Command: $command"; #Output is: Command: ls

b=`$command`;
echo $b; #Output is: public_html REV test... (command worked successfully)

However, when I try something a bit more complicated, it fails. For example, if I make

command="ls | grep -c '^'";

The output is:

Command: ls | grep -c '^'
ls: cannot access |: No such file or directory
ls: cannot access grep: No such file or directory
ls: cannot access '^': No such file or directory

How could I store such a command (with pipes/multiple commands) in a variable for later use?

Curative

Use a function! – Lagerkvist

See this post: Why should eval be avoided in Bash, and what should I use instead?. – Anallise

See also Why does shell ignore quoting characters in arguments passed to it through variables? and the similar mywiki.wooledge.org/BashFAQ/050 – Anthotaxy

Answer 1

Use eval:

x="ls | wc"
eval "$x"
y=$(eval "$x")
echo "$y"

Noteworthy

$(...) is now recommended instead of backticks. y=$(eval $x) mywiki.wooledge.org/BashFAQ/082 – Velocipede

while that has been confusing at start, it's running as expected in both ways. eval is fine. Thank you. – Technic

eval is an acceptable practice only if you trust your variables' contents. If you're running, say, x="ls $name | wc" (or even x="ls '$name' | wc"), then this code is a fast track to injection or privilege escalation vulnerabilities if that variable can be set by someone with less privileges. (Iterating over all subdirectories in /tmp, for instance? You'd better trust every single user on the system to not make one called $'/tmp/evil-$(rm -rf $HOME)\'$(rm -rf $HOME)\'/'). – Hermia

eval is a huge bug magnet that should never be recommended without a warning about the risk of unexpected parsing behavior (even without malicious strings, as in @CharlesDuffy's example). For example, try x='echo $(( 6 * 7 ))' and then eval $x. You might expect that to print "42", but it probably won't. Can you explain why it doesn't work? Can you explain why I said "probably"? If the answers to those questions aren't obvious to you, you should never touch eval. – Emilie

@GordonDavisson why... I have completely no idea (and I do follow what you said.. i took away eval in my script!) – Expeditionary

@Student, try running set -x beforehand to log the commands run, which will make it easier to see what's happening. – Hermia

@CharlesDuffy that is a spectacular trick! I am pretty new to this, and did not learn it systematically. I think I should study some theory behind it for a while until I start making scripts again.. any good suggestions? – Expeditionary

The POSIX sh specification is the canonical source. Beyond that, I'm fond of the Wooledge wiki, which beyond the BashGuide also hosts background such as the BashParser page. – Hermia

@Expeditionary I'd also recommend shellcheck.net for pointing out common mistakes (and bad habits you shouldn't pick up). – Emilie

Answer 2

Do not use eval! It has a major risk of introducing arbitrary code execution.

BashFAQ-50 - I'm trying to put a command in a variable, but the complex cases always fail.

Put it in an array and expand all the words with double-quotes "${arr[@]}" to not let the IFS split the words due to Word Splitting.

cmdArgs=()
cmdArgs=('date' '+%H:%M:%S')

and see the contents of the array inside. The declare -p allows you see the contents of the array inside with each command parameter in separate indices. If one such argument contains spaces, quoting inside while adding to the array will prevent it from getting split due to Word-Splitting.

declare -p cmdArgs
declare -a cmdArgs='([0]="date" [1]="+%H:%M:%S")'

and execute the commands as

"${cmdArgs[@]}"
23:15:18

(or) altogether use a bash function to run the command,

cmd() {
   date '+%H:%M:%S'
}

and call the function as just

cmd

POSIX sh has no arrays, so the closest you can come is to build up a list of elements in the positional parameters. Here's a POSIX sh way to run a mail program

# POSIX sh
# Usage: sendto subject address [address ...]
sendto() {
    subject=$1
    shift
    first=1
    for addr; do
        if [ "$first" = 1 ]; then set --; first=0; fi
        set -- "$@" --recipient="$addr"
    done
    if [ "$first" = 1 ]; then
        echo "usage: sendto subject address [address ...]"
        return 1
    fi
    MailTool --subject="$subject" "$@"
}

Note that this approach can only handle simple commands with no redirections. It can't handle redirections, pipelines, for/while loops, if statements, etc

Another common use case is when running curl with multiple header fields and payload. You can always define args like below and invoke curl on the expanded array content

curlArgs=('-H' "keyheader: value" '-H' "2ndkeyheader: 2ndvalue")
curl "${curlArgs[@]}"

Another example,

payload='{}'
hostURL='http://google.com'
authToken='someToken'
authHeader='Authorization:Bearer "'"$authToken"'"'

now that variables are defined, use an array to store your command args

curlCMD=(-X POST "$hostURL" --data "$payload" -H "Content-Type:application/json" -H "$authHeader")

and now do a proper quoted expansion

curl "${curlCMD[@]}"

Sailmaker

This does not work for me, I have tried Command=('echo aaa | grep a') and "${Command[@]}", hoping it runs literally the command echo aaa | grep a. It doesn't. I wonder if there's a safe way replacing eval, but seems that each solution that has the same force as eval could be dangerous. Isn't it? – Expeditionary

In short, how does this work if the original string contains a pipe '|'? – Expeditionary

@Student, if your original string contains a pipe, then that string needs to go through the unsafe parts of the bash parser to be executed as code. Don't use a string in that case; use a function instead: Command() { echo aaa | grep a; } -- after which you can just run Command, or result=$(Command), or the like. – Hermia

@CharlesDuffy problem is it fails if I want to do Command() {$1}; Command "echo aaa | grep a" – Expeditionary

@Student, right; but that fails intentionally, because what you're asking to do is inherently insecure. – Hermia

@Student: I've added a note at the last to mention it doesn't work under certain conditions – Sailmaker

with no redirections. It can't handle redirections, pipelines, for/while loops, if statements,, but there is an if: if [ "$first" = 1 ]; then.. – Caress

@Caress It's saying that you can't pass in an if statement to sendto; the fact that the code of the implementation contains an if statement and some other logic is unrelated to this restriction. – Anthotaxy

@tripleee, you mean pass logic as param to sendto() or as $1? – Caress

Yes, exactly. For example, you can't sendto if true; then echo poo; fi because it looks like you are sending if true, which in isolation is obviously a syntax error, and the following statements are unrelated to the sendto call. – Anthotaxy

@tripleee: a bit kudos for offering the bounty, wish I could share it with you and other useful contributors ;) – Sailmaker

Answer 3

var=$(echo "asdf")
echo $var
# => asdf

Using this method, the command is immediately evaluated and its return value is stored.

stored_date=$(date)
echo $stored_date
# => Thu Jan 15 10:57:16 EST 2015
# (wait a few seconds)
echo $stored_date
# => Thu Jan 15 10:57:16 EST 2015

The same with backtick

stored_date=`date`
echo $stored_date
# => Thu Jan 15 11:02:19 EST 2015
# (wait a few seconds)
echo $stored_date
# => Thu Jan 15 11:02:19 EST 2015

Using eval in the $(...) will not make it evaluated later:

stored_date=$(eval "date")
echo $stored_date
# => Thu Jan 15 11:05:30 EST 2015
# (wait a few seconds)
echo $stored_date
# => Thu Jan 15 11:05:30 EST 2015

Using eval, it is evaluated when eval is used:

stored_date="date" # < storing the command itself
echo $(eval "$stored_date")
# => Thu Jan 15 11:07:05 EST 2015
# (wait a few seconds)
echo $(eval "$stored_date")
# => Thu Jan 15 11:07:16 EST 2015
#                     ^^ Time changed

In the above example, if you need to run a command with arguments, put them in the string you are storing:

stored_date="date -u"
# ...

For Bash scripts this is rarely relevant, but one last note. Be careful with eval. Eval only strings you control, never strings coming from an untrusted user or built from untrusted user input.

Memorialist

This does not solve the original problem where the command contains a pipe '|'. – Expeditionary

@Nate, note that eval $stored_date may be fine enough when stored_date only contains date, but eval "$stored_date" is much more reliable. Run str=$'printf \' * %s\\n\' *'; eval "$str" with and without the quotes around the final "$str" for an example. :) – Hermia

@CharlesDuffy Thanks, I forgot about quoting. I'll bet my linter would have complained had I bothered to run it. – Memorialist

Tangentially, that's a useless echo – Anthotaxy

Answer 4

For bash, store your command like this:

command="ls | grep -c '^'"

Run your command like this:

echo $command | bash

Legpull

Not sure but perhaps this way of running the command has the same risks that the use of 'eval' has. – Legpull

In addition, you are wrecking the contents of the variable by not quoting it when you echo it. If command was the string cd /tmp && echo * it will echo the files in the current directory, not in /tmp. See also When to wrap quotes around a shell variable – Anthotaxy

to be clear echo "$command" | bash – Nereid

Answer 5

Not sure why so many answers make it complicated! use alias [command] 'string to execute' example:

alias dir='ls -l'

./dir
[pretty list of files]

Atmo

There are many problems with aliases. For one thing, they do not expand in scripts, which is obviously a bummer if you are writing a script (and thus your question is on-topic on Stack Overflow). For another, you can't pass positional arguments to aliases, just have them eat up the rest of your command line with no access to it. Also, the parsing rules are finicky, so you can't enable alias expansion and use the alias in the same command line. Bottom line, use a function, like everyone was telling you all along. – Anthotaxy

Answer 6

I faced this problem with the following command:

awk '{printf "%s[%s]\n", $1, $3}' "input.txt"

I need to build this command dynamically:

The target file name input.txt is dynamic and may contain space.

The awk script inside {} braces printf "%s[%s]\n", $1, $3 is dynamic.

Challenge:

1. Avoid extensive quote escaping logic if there are many " inside the awk script.
2. Avoid parameter expansion for every $ field variable.

The solutions bellow with eval command and associative arrays do not work. Due to bash variable expansions and quoting.

Solution:

Build bash variable dynamically, avoid bash expansions, use printf template.

 # dynamic variables, values change at runtime.
 input="input file 1.txt"
 awk_script='printf "%s[%s]\n" ,$1 ,$3'

 # static command template, preventing double-quote escapes and avoid variable  expansions.
 awk_command=$(printf "awk '{%s}' \"%s\"\n" "$awk_script" "$input")
 echo "awk_command=$awk_command"

 awk_command=awk '{printf "%s[%s]\n" ,$1 ,$3}' "input file 1.txt"

Executing variable command:

bash -c "$awk_command"

Alternative that also works

bash << $awk_command

Wayward

Answer 7

I tried various different methods:

printexec() {
  printf -- "\033[1;37m$\033[0m"
  printf -- " %q" "$@"
  printf -- "\n"
  eval -- "$@"
  eval -- "$*"
  "$@"
  "$*"
}

Output:

$ printexec echo  -e "foo\n" bar
$ echo -e foo\\n bar
foon bar
foon bar
foo
 bar
bash: echo -e foo\n bar: command not found

As you can see, only the third one, "$@" gave the correct result.

Carvajal

What is the explanation for that? Why only the third one? Please respond by editing (changing) your answer, not here in comments (without "Edit:", "Update:", or similar - the answer should appear as if it was written today). – Shroff

Not sure I care enough to investigate the intricacies of eval. IMO one of those 2 should have worked. – Carvajal

Answer 8

Be careful registering an order with the: X=$(Command)

This one is still executed. Even before being called. To check and confirm this, you can do:

echo test;
X=$(for ((c=0; c<=5; c++)); do
sleep 2;
done);
echo note the 5 seconds elapsed

Glasshouse

This doesn't seem to be an answer to the actual question here, and should probably be a comment instead (if even that). – Anthotaxy

What do you mean by "registering an order"? Can you elaborate? – Shroff

Answer 9

#!/bin/bash
#Note: this script works only when u use Bash. So, don't remove the first line.

TUNECOUNT=$(ifconfig |grep -c -o tune0) #Some command with "Grep".
echo $TUNECOUNT                         #This will return 0 
                                    #if you don't have tune0 interface.
                                    #Or count of installed tune0 interfaces.

Succumb

This stores the static string output from the command in a variable, not the command itself. – Anthotaxy

grep -c -o is not entirely portable; you would perhaps expect it to return the number of actual number of occurrences of the search expression, but at least GNU grep does not do that (it's basically equivalent to grep -c without the -o). – Anthotaxy

The Bash-only comment is weird; there is nothing in this simple script which isn't compatible with any Bourne-family shell. – Anthotaxy

Answer 10

First of all, there are functions for this. But if you prefer variables then your task can be done like this:

$ cmd=ls

$ $cmd # works
file  file2  test

$ cmd='ls | grep file'

$ $cmd # not works
ls: cannot access '|': No such file or directory
ls: cannot access 'grep': No such file or directory
 file

$ bash -c $cmd # works
file  file2  test

$ bash -c "$cmd" # also works
file
file2

$ bash <<< $cmd
file
file2

$ bash <<< "$cmd"
file
file2

Or via a temporary file

$ tmp=$(mktemp)
$ echo "$cmd" > "$tmp"
$ chmod +x "$tmp"
$ "$tmp"
file
file2

$ rm "$tmp"

Horsecar

I suppose many people didn't notice the "First of all there are functions for this" you mentioned that is a correct pointer to the right direction IMHO: "Variables hold data. Functions hold code" – Hobby

Your bash -c $cmd ad bash -c "$cmd" cases do two VERY different things! Try it each way with cmd='echo "$(pwd)"; cd test2; echo "$(pwd)"' To see why, do 'bash -x -c $cmd'. – Addam

Also: Functions are not a solution, They may be an alternative in some cases, but you have to construct the function! If you get the pieces of $cd as a series of $1, $2, etc., have fun turning it into a function, and then have even more fun not using eval to make use of it in another shell. – Addam

Answer 11

As you don't specify any scripting language, I would recommand tcl, the Tool Command Language for this kind of purpose.

Then in the first line, add the appropriate shebang:

#!/usr/local/bin/tclsh

with appropriate location you can retrieve with which tclsh.

In tcl scripts, you can call operating system commands with exec.

Oshea

Answer 12

It is not necessary to store commands in variables even as you need to use it later. Just execute it as per normal. If you store in variables, you would need some kind of eval statement or invoke some unnecessary shell process to "execute your variable".

Medina

The command I will store will depend on options I send in, so instead of having tons of conditional statements in the bulk of my program it's a lot easier to store the command I need for later use. – Curative

@Benjamin, then at least store the options as variables, and not the command. eg var='*.txt'; find . -name "$var" – Medina