How to write bash function to print and run command when the command has arguments with spaces or things to be expanded

Question

In Bash scripts, I frequently find this pattern useful, where I first print the command I'm about to execute, then I execute the command:

echo 'Running this cmd: ls -1 "$HOME/temp/some folder with spaces'
ls -1 "$HOME/temp/some folder with spaces"

echo 'Running this cmd: df -h'
df -h

# etc.

Notice the single quotes in the echo command to prevent variable expansion there! The idea is that I want to print the cmd I'm running, exactly as I will type and run the command, then run it!

How do I wrap this up into a function?

Wrapping the command up into a standard bash array, and then printing and calling it, like this, sort-of works:

# Print and run the passed-in command
# USAGE:
#       cmd_array=(ls -a -l -F /)
#       print_and_run_cmd cmd_array
# See:
# 1. My answer on how to pass regular "indexed" and associative arrays by reference:
#    https://mcmap.net/q/41290/-passing-arrays-as-parameters-in-bash and
# 1. My answer on how to pass associative arrays: https://mcmap.net/q/41561/-how-to-pass-an-associative-array-as-argument-to-a-function-in-bash
print_and_run_cmd() {
    local -n array_reference="$1"
    echo "Running cmd:  ${cmd_array[@]}"
    # run the command by calling all elements of the command array at once
    ${cmd_array[@]}
}

For simple commands like this it works fine:

Usage:

cmd_array=(ls -a -l -F /)
print_and_run_cmd cmd_array

Output:

Running cmd:  ls -a -l -F /
(all output of that cmd is here)

But for more-complicated commands it is broken!:

Usage:

cmd_array=(ls -1 "$HOME/temp/some folder with spaces")
print_and_run_cmd cmd_array

Desired output:

Running cmd: ls -1 "$HOME/temp/some folder with spaces"
(all output of that command should be here)

Actual Output:

Running cmd:  ls -1 /home/gabriel/temp/some folder with spaces
ls: cannot access '/home/gabriel/temp/some': No such file or directory
ls: cannot access 'folder': No such file or directory
ls: cannot access 'with': No such file or directory
ls: cannot access 'spaces': No such file or directory

The first problem, as you can see, is that $HOME got expanded in the Running cmd: line, when it shouldn't have, and the double quotes around that path argument were removed, and the 2nd problem is that the command doesn't actually run.

How do I fix these 2 problems?

References:

1. my bash demo program where I have this print_and_run_cmd function: https://github.com/ElectricRCAircraftGuy/eRCaGuy_hello_world/blob/master/bash/argument_parsing__3_advanced__gen_prog_template.sh
2. where I first documented how to pass bash arrays by reference, as I do in that function:
   1. Passing arrays as parameters in bash
   2. How to pass an associative array as argument to a function in Bash?

Follow-up question:

1. Bash: how to print and run a cmd array which has the pipe operator, |, in it

Airt

The variable expansion happens before the function call, so there's no way how the function can access the original command as written. – Speller

@choroba, any way you know to rewrite the command to allow it to do what I want, printing without the expansion? – Airt

Seems to me that cmd_array=(ls -1 "$HOME/temp/some folder with spaces") should be cmd_array=('ls -1 "$HOME/temp/some folder with spaces"') because cmd_array=(ls -1 "$HOME/temp/some folder with spaces") runs before it runs in print_and_run_cmd(). The only addition being a glitch ' to keep things from running or expanding before `print_and_run_cmd() is executed – Katakana

Answer 1

If you've got Bash version 4.4 or later, this function may do what you want:

function print_and_run_cmd
{
    local PS4='Running cmd: '
    local -
    set -o xtrace

    "$@"
}

For example, running

print_and_run_cmd echo 'Hello World!'

outputs

Running cmd: echo 'Hello World!'
Hello World!

• local PS4='Running cmd: ' sets a prefix for commands printed by the shell when the xtrace option is on. The default is + . Localizing it means that the previous value of PS4 is automatically restored when the function returns.

• local - causes any changes to shell options to be reverted automatically when the function returns. In particular, it causes the set -o xtrace on the next line to be automatically undone when the function returns. Support for local - was added in Bash 4.4.

  From man bash, under the local [option] [name[=value] ... | - ] section (emphasis added):

  If name is -, the set of shell options is made local to the function in which local is invoked: shell options changed using the set builtin inside the function are restored to their original values when the function returns.

• set -o xtrace (which is equivalent to set -x) causes the shell to print commands, preceded by the expanded value of PS4, before running them.

  See help set.

Boll

I love this answer, and its output! Running bash --version on Ubuntu 18.04 shows that it comes with GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu). So, this answer should work on Ubuntu 18.04 or later! – Airt

I've added my own answer, based on your answer, here. – Airt

If you feel so inclined, feel free to take a look at my follow-up question too. It's also pasted at the bottom of my question here. – Airt

local -! I need this, thanks – Creasy

Answer 2

You can achieve what you want with DEBUG trap :

#!/bin/bash
 
set -T
trap 'test "$FUNCNAME" = print_and_run_cmd || trap_saved_command="${BASH_COMMAND}"' DEBUG
print_and_run_cmd(){
    echo "Running this cmd: ${trap_saved_command#* }"
    "$@"
}
outer(){
    print_and_run_cmd ls -1 "$HOME/temp/some folder with spaces"
}
outer
# output ->
# Running this cmd: ls -1 "$HOME/temp/some folder with spaces"
# ...

Wooldridge

Upvoted, but place the call to print_and_run_cmd into another function, ex: main, and I think it breaks it, causing it to print out this: Running this cmd: main. That is my scenario. – Airt

@GabrielStaples Seems to work with set -T – Wooldridge

Thanks for finding the set -T option. That does work. However, it also means that if I move to this style of command call: cmd_array=(ls -1 "$HOME/temp/some folder with spaces") and then print_and_run_cmd "${cmd_array[@]}", then my output is this: Running this cmd: "${cmd_array[@]}" instead of this: Running this cmd: ls -1 "$HOME/temp/some folder with spaces". That's unfortunate. It seems no matter what I do I can't make this work quite how I want it to. I've also added a link to a follow-up question to the bottom of my question. – Airt

Bash feels a bit like a nightmare to me. Aaaah! :) – Airt

Answer 3

Check your scripts with shellcheck:

Line 2:
    local -n array_reference="$1"
             ^-- SC2034 (warning): array_reference appears unused. Verify use (or export if used externally).
 
Line 3:
    echo "Running cmd:  ${cmd_array[@]}"
                        ^-- SC2145 (error): Argument mixes string and array. Use * or separate argument.
                        ^-- SC2154 (warning): cmd_array is referenced but not assigned.
 
Line 5:
    ${cmd_array[@]}
    ^-- SC2068 (error): Double quote array expansions to avoid re-splitting elements.

You might want to research https://github.com/koalaman/shellcheck/wiki/SC2068 . We fix all errors and we get:

print_and_run_cmd() {
    local -n array_reference="$1"
    echo "Running cmd:  ${array_reference[*]}"
    # run the command by calling all elements of the command array at once
    "${array_reference[@]}"
}

For me it's odd to pass an array by reference in this case. I would pass the actual values. I often do:

prun() {
   # in the style of set -x
   # print to stderr, so output can be captured
   echo "+ $*" >&2
   # or echo "+ ${*@Q}" >&2
   # or echo "+$(printf " %q" "$@")" >&2
   # or echo "+$(/bin/printf " %q" "$@")" >&2
   "$@"
}
prun "${cmd_array[@]}"

How do I fix these 2 problems?

Incorporate into your workflow linters, formatters and static analysis tools, like shellcheck, and check the problems they point out.

And quote variable expansion. It's "${array[@]}".

Transceiver

Thanks. This is useful. I was using cmd_array, accidentally, instead of array_reference, inside print_and_run_cmd. Fixing that was step 1. Step 2 was to add double quotes around my call to the cmd to make it "${array_reference[@]}". But, my sample output is Running cmd: ls -1 /home/gabriel/temp/some folder with spaces instead of Running cmd: ls -1 "$HOME/temp/some folder with spaces". How can I get the latter? – Airt

How can I get the latter? echo "running cmd: ${*@Q}" or echo "running cmd:$(printf " %q" "$@")" or echo "running cmd:$(/bin/printf " %q" "$@")". For array, respectively ${array[*]@Q} or printf " %q" "${array[@]}". – Transceiver

Where can I learn what this *@Q syntax means? – Airt

From gnu.org/software/bash/manual/html_node/… ${parameter@operator} – Transceiver

For these commands (echo "running cmd: ${*@Q}" echo "running cmd:$(printf " %q" "$@")" echo "running cmd:$(/usr/bin/printf " %q" "$@")" ) I get this respective output: running cmd: 'ls' '-1' '/home/gabriel/temp/some folder with spaces' running cmd: ls -1 /home/gabriel/temp/some\ folder\ with\ spaces running cmd: ls -1 '/home/gabriel/temp/some folder with spaces'. All 3 still have expanded $HOME and none show it like Running cmd: ls -1 "$HOME/temp/some folder with spaces". – Airt

HOME is expanded in the array at line cmd_array=(ls -1 "$HOME/te.."). After that, it's expanded. You can store the literal string $HOME inside the array and replace it in each array value before execution, you could replace the result of $HOME by the string $HOME when printing. For both, you have to write code yourself. – Transceiver

Thanks for the help. You clearly know a lot more about bash than I do, and I have a lot to study. Meanwhile, can you think of a way to achieve what I'm trying to achieve? Feel free to alter how I store the command, pass the command, print the command, and call the command. Maybe I should be passing the cmd in a single-quoted string, then printing it, then parsing it into an array from there, then calling it? – Airt

Thanks for all your help; here's a follow-up question if you have a few minutes: Bash: how to print and run a cmd array which has the pipe operator, |, in it – Airt

Answer 4

I really like @pjh's answer, so I've marked it as correct. It doesn't fully answer my original question though, so if another answer comes along that does, I may have to change that. Anyway, see @pjh's answer or a full explanation of how the below code works, and what all those lines mean. I've helped edit that answer with some of the sources from man bash and help set.

I'd like to change the formatting and provide some more examples, however, to show that variable expansion does take place within the command. I'd also like to provide one version which passes by reference, and one which does not, so you can choose the call style which you like best.

Here are my examples, showing both call styles (print_and_run1 cmd_array and print_and_run2 "${cmd_array[@]}"):

#!/usr/bin/env bash

# Print and run the passed-in command, which is passed in as an 
# array **by reference**.
# See here for a full explanation: https://mcmap.net/q/429197/-how-to-write-bash-function-to-print-and-run-command-when-the-command-has-arguments-with-spaces-or-things-to-be-expanded
# USAGE:
#       cmd_array=(ls -a -l -F /)
#       print_and_run1 cmd_array
print_and_run1() {
    local -n array_reference="$1"
    local PS4='Running cmd:  '
    local -
    set -o xtrace
    # Call the cmd
    "${array_reference[@]}"
}

# Print and run the passed-in command, which is passed in as members
# of an array **by value**.
# See here for a full explanation: https://mcmap.net/q/429197/-how-to-write-bash-function-to-print-and-run-command-when-the-command-has-arguments-with-spaces-or-things-to-be-expanded
# USAGE:
#       cmd_array=(ls -a -l -F /)
#       print_and_run2 "${cmd_array[@]}"
print_and_run2() {
    local PS4='Running cmd:  '
    local -
    set -o xtrace
    # Call the cmd
    "$@"
}

cmd_array=(ls -1 "$HOME/temp/some folder with spaces")

print_and_run1 cmd_array
echo ""
print_and_run2 "${cmd_array[@]}"
echo ""

Sample run and output:

eRCaGuy_hello_world/bash$ ./print_and_run.sh 
Running cmd:  ls -1 '/home/gabriel/temp/some folder with spaces'
file1.txt
file2.txt

Running cmd:  ls -1 '/home/gabriel/temp/some folder with spaces'
file1.txt
file2.txt

Airt

Answer 5

This seems to work too:

print_and_run_cmd() {
    echo "Running cmd:  $1"
    eval "$cmd"
}

cmd='ls -1 "$HOME/temp/some folder with spaces"'
print_and_run_cmd "$cmd"

Output:

Running cmd:  ls -1 "$HOME/temp/some folder with spaces"
(result of running the cmd is here)

But now the problem is, if I want to print an expanded version of the cmd too, to verify that part worked properly, I can't, or at least, don't know how.

Airt

t=`eval echo $cmd` .... echo $t Will print the expanded path instead of $HOME – Katakana