How to echo shell commands as they are executed

Question

In a shell script, how do I echo all shell commands called and expand any variable names?

For example, given the following line:

ls $DIRNAME

I would like the script to run the command and display the following

ls /full/path/to/some/dir

The purpose is to save a log of all shell commands called and their arguments. Is there perhaps a better way of generating such a log?

Interrupter

See also (duplicate): How can I print each command before executing? – Inadvertence

Answer 1

set -x or set -o xtrace expands variables and prints a little + sign before the line.

set -v or set -o verbose does not expand the variables before printing.

Use set +x and set +v to turn off the above settings.

On the first line of the script, one can put #!/bin/sh -x (or -v) to have the same effect as set -x (or -v) later in the script.

The above also works with /bin/sh.

See the bash-hackers' wiki on set attributes, and on debugging.

$ cat shl
#!/bin/bash                                                                     

DIR=/tmp/so
ls $DIR

$ bash -x shl 
+ DIR=/tmp/so
+ ls /tmp/so
$

Hifi

If you also want to see which numbered line is being executed see https://mcmap.net/q/46478/-how-to-show-line-number-when-executing-bash-script – Dacia

what if I want to color the command when echoing to differentiate the command and its results output ? – Reprove

(The ABS has a long history of being unresponsive to requests that they correct bad-practice examples, to the point of driving longstanding community members to create competing resources; moved links over to the bash-hackers' wiki -- the Wooledge wiki or the official bash manual would also be better resources). – Potful

@LewisChan : you can add a colored static or dynamic prefix, f.e. timestamp, to your commands, see https://mcmap.net/q/46479/-bash-highlight-command-before-execution-set-x . – Snooze

...and if you are using iTerm2, @LewisChan, then for non-scripts you can color the commands differently than the outputs with this apple.stackexchange.com/a/252001 . – Snooze

@AndreasDietrich Thanks. You'd make my day if you actually found a way to implement my joke question. Preferably with some powerful cli tools and only a few lines of code. – Ayotte

bash -x foo.sh was the crux of what I needed. (Posting as a comment because it wasn't immediately apparent whether that would work without modifying the script itself; it did. ) – Haskins

set -o verbose does nothing for me (Arch Linux where sh --version yields GNU bash, version 5.1.8) – Chavarria

The + prefix can be disabled with PS4="\000". By default this output goes to stderr. To output to stdout, use BASH_XTRACEFD=1. – Parkerparkhurst

Note that the bash-hackers' wiki no longer exists. – Belong

@Hifi any idea how to print piped commands better? – Kao

Answer 2

set -x will give you what you want.

Here is an example shell script to demonstrate:

#!/bin/bash
set -x #echo on

ls $PWD

This expands all variables and prints the full commands before output of the command.

Output:

+ ls /home/user/
file1.txt file2.txt

Danford

Using the word "verbose" that way doesn't accomplish anything. You can do set -o verbose or set -v (only "verbose") or set -o xtrace or set -x (only "xtrace") or set -xv (both) or set -o xtrace -o verbose (both). – Berzelius

this works good, but be aware that the "verbose" overwrites $1 – Screwball

Answer 3

I use a function to echo and run the command:

#!/bin/bash
# Function to display commands
exe() { echo "\$ $@" ; "$@" ; }

exe echo hello world

Which outputs

$ echo hello world
hello world

For more complicated commands pipes, etc., you can use eval:

#!/bin/bash
# Function to display commands
exe() { echo "\$ ${@/eval/}" ; "$@" ; }

exe eval "echo 'Hello, World!' | cut -d ' ' -f1"

Which outputs

$  echo 'Hello, World!' | cut -d ' ' -f1
Hello

Rockies

Not many votes for this answer. Is there a reason it's a bad idea? Worked for me, and seems to be exactly what I'm looking for... – Galliwasp

This is the best answer if you don't want every command printed. It avoids the ++ set +x output when turned off, as well as looking cleaner. For just a single statement or two, though, bhassel's answer using a subshell is the most convenient. – Doctorate

This is what I'm looking for! Not set +x, it affects all commands, which is too much! – Predikant

A major downside to this is that the output loses the quoting information. You can't differentiate between cp "foo bar" baz and cp foo "bar baz", for example. So it's good for displaying progress information to a user; less so for debugging output or recording reproducible commands. Different use cases. In zsh, you can preserve quoting with the :q modifier: exe() { echo '$' "${@:q}" ; "$@" ; } – Juli

This does not seem to work with shell builtins, e.g. exe foo=bar; echo $foo will not work. – Litt

It seems to me a great solution. It can eventually be integrated with a | tee -a $LOGFILE to printout do std output and the log file as well. – Buller

So the major downside above is only related to the output right? The actual command formatting for what is actually executed is not affected? – Crappie

I love this solution and have been using it for a while. However, it seems to fail at complex commands: e.g. exe (cut -d ' ' -f9,10 --complement ${folder}/hog_crop_px.txt) | paste - ${folder}/lbp_crop_px.txt > ${folder}/crop_px.tx gives me syntax error near unexpected token cut'` even though it executes perfectly without the exe() command. Any suggestion? – Gehlenite

@Gehlenite you could use exe eval"(cut -d ' ' -f9,10 --complement ${folder}/hog_crop_px.txt) | paste - ${folder}/lbp_crop_px.txt > ${folder}/crop_px.tx" ----- OR ----- exe cut -d ' ' -f9,10 --complement ${folder}/hog_crop_px.txt | paste - ${folder}/lbp_crop_px.txt > ${folder}/crop_px.tx – Rockies

Just modified the program so that exe eval 'foo=bar; echo $foo' will now work – Rockies

@Rockies I am trying your new function, and it seems like it should work okay with eval. However, I'm sure I've tried using your original function with and without brackets before I posted here and your second example was not working. – Gehlenite

I don't like this answer. There are lots of edge cases where what you see is not what you get (especially with whitespace, quotes, escaped characters, variable/expression substitutions, etc), so don't blindly paste the echoed command into a terminal and assume it will run the same way. Also, the second technique is just a hack, and will strip out other instances of the word eval from your command. So don't expect it to work properly on exe eval "echo 'eval world'"! – Debus

a drawback is that you cannot use this exe function in subshell commands like: VAR=$(exe echo "hello world"); echo $VAR The value of $VAR will be the echoed command plus the result of the command. like: 'echo hello world hello world' – Injure

I have not tested it but I think there is a drawback to it. If your command had something like "echo $RANDOM" where $RANDOM is a function to get a random number in bash. Your echo output and real command will be different. – Boatbill

No matter what I tried, I could not get the double space after the $ to be a single space using variable substitution. So instead I used sed: run() { echo "$" "$(echo "$@" | sed -e 's/eval //' )"; "$@"; } – Introversion

The last output is wrong, should be 'Hello,' Otherwise you should split on comma ',' – Incandescent

Answer 4

You can also toggle this for select lines in your script by wrapping them in set -x and set +x, for example,

#!/bin/bash
...
if [[ ! -e $OUT_FILE ]];
then
   echo "grabbing $URL"
   set -x
   curl --fail --noproxy $SERV -s -S $URL -o $OUT_FILE
   set +x
fi

Donau

But, script prints set +x too – Dolor

Tips: We can use { set +x; } &> /dev/null to silence the set +x output. – Torpedo

nice tip @Torpedo – Montpelier

Answer 5

shuckc's answer for echoing select lines has a few downsides: you end up with the following set +x command being echoed as well, and you lose the ability to test the exit code with $? since it gets overwritten by the set +x.

Another option is to run the command in a subshell:

echo "getting URL..."
( set -x ; curl -s --fail $URL -o $OUTFILE )

if [ $? -eq 0 ] ; then
    echo "curl failed"
    exit 1
fi

which will give you output like:

getting URL...
+ curl -s --fail http://example.com/missing -o /tmp/example
curl failed

This does incur the overhead of creating a new subshell for the command, though.

Acth

Nice way to avoid the ++ set +x output. – Doctorate

Even better: replace if [ $? -eq 0 ] with if (set -x; COMMAND). – Piercing

In my question someone recommended this cool solution: superuser.com/questions/806599/… – Investigator

Answer 6

According to TLDP's Bash Guide for Beginners: Chapter 2. Writing and debugging scripts:

2.3.1. Debugging on the entire script

$ bash -x script1.sh

...

There is now a full-fledged debugger for Bash, available at SourceForge. These debugging features are available in most modern versions of Bash, starting from 3.x.

2.3.2. Debugging on part(s) of the script

set -x            # Activate debugging from here
w
set +x            # Stop debugging from here

...

Table 2-1. Overview of set debugging options

    Short  | Long notation | Result
    -------+---------------+--------------------------------------------------------------
    set -f | set -o noglob | Disable file name generation using metacharacters (globbing).
    set -v | set -o verbose| Prints shell input lines as they are read.
    set -x | set -o xtrace | Print command traces before executing command.

...

Alternatively, these modes can be specified in the script itself, by adding the desired options to the first line shell declaration. Options can be combined, as is usually the case with UNIX commands:

#!/bin/bash -xv

Kolivas

Answer 7

Another option is to put "-x" at the top of your script instead of on the command line:

$ cat ./server
#!/bin/bash -x
ssh user@server

$ ./server
+ ssh user@server
user@server's password: ^C
$

Bagnio

Note that this doesn't seem to work exactly the same between ./myScript and bash myScript. Still a good thing to point out, thanks. – Koch

Answer 8

You can execute a Bash script in debug mode with the -x option.

This will echo all the commands.

bash -x example_script.sh

# Console output
+ cd /home/user
+ mv text.txt mytext.txt

You can also save the -x option in the script. Just specify the -x option in the shebang.

######## example_script.sh ###################
#!/bin/bash -x

cd /home/user
mv text.txt mytext.txt

##############################################

./example_script.sh

# Console output
+ cd /home/user
+ mv text.txt mytext.txt

Screen

Also bash -vx will do the same but without variable interpolation – Stone

This is nice, but a bit more hardcore than I wanted. It seems to "descend" into all the commands run by my top-level script. I really just wanted the commands of my top-level script to be echoed, not absolutely everything bash runs. – Hernandez

Answer 9

Combining all the answers I found this to be the best, simplest

#!/bin/bash
# https://mcmap.net/q/45406/-how-to-echo-shell-commands-as-they-are-executed
exe(){
    set -x
    "$@"
    { set +x; } 2>/dev/null
}
# example
exe go generate ./...

{ set +x; } 2>/dev/null from https://mcmap.net/q/46481/-bash-set-x-without-it-being-printed

If the exit status of the command is needed, as mentioned here

Use

{ STATUS=$?; set +x; } 2>/dev/null

And use the $STATUS later like exit $STATUS at the end

A slightly more useful one

#!/bin/bash
# https://mcmap.net/q/45406/-how-to-echo-shell-commands-as-they-are-executed
_exe(){
    [ $1 == on  ] && { set -x; return; } 2>/dev/null
    [ $1 == off ] && { set +x; return; } 2>/dev/null
    echo + "$@"
    "$@"
}
exe(){
    { _exe "$@"; } 2>/dev/null
}

# examples
exe on # turn on same as set -x
echo This command prints with +
echo This too prints with +
exe off # same as set +x
echo This does not

# can also be used for individual commands
exe echo what up!

Phares

Great examples. I would add #!/bin/bash to the top of both scripts. – Gehring

Answer 10

Type "bash -x" on the command line before the name of the Bash script. For instance, to execute foo.sh, type:

bash -x foo.sh

Densmore

Answer 11

For zsh, echo

setopt VERBOSE

And for debugging,

setopt XTRACE

Grimsley

Thanks! This is interesting. verbose can be in lowercase. To remove it, use unsetopt instead of setopt. Example: unsetopt verbose. PS: This doesn't work in scripts though. – Hadley

Answer 12

To allow for compound commands to be echoed, I use eval plus Soth's exe function to echo and run the command. This is useful for piped commands that would otherwise only show none or just the initial part of the piped command.

Without eval:

exe() { echo "\$ $@" ; "$@" ; }
exe ls -F | grep *.txt

Outputs:

$
file.txt

With eval:

exe() { echo "\$ $@" ; "$@" ; }
exe eval 'ls -F | grep *.txt'

Which outputs

$ exe eval 'ls -F | grep *.txt'
file.txt

Oversell

Answer 13

For csh and tcsh, you can set verbose or set echo (or you can even set both, but it may result in some duplication most of the time).

The verbose option prints pretty much the exact shell expression that you type.

The echo option is more indicative of what will be executed through spawning.

---

http://www.tcsh.org/tcsh.html/Special_shell_variables.html#verbose

http://www.tcsh.org/tcsh.html/Special_shell_variables.html#echo

---

Special shell variables

verbose If set, causes the words of each command to be printed, after history substitution (if any). Set by the -v command line option.

echo If set, each command with its arguments is echoed just before it is executed. For non-builtin commands all expansions occur before echoing. Builtin commands are echoed before command and filename substitution, because these substitutions are then done selectively. Set by the -x command line option.

Anselma

How do you disable echo/verbose once you set it? – Reductase

Answer 14

Following the answer of Soth.

It is possible to create a markdown output without highlight (no language given)

set -x
exe() { echo "\`\$\$  ${@/eval/} \`" ; "$@" ; }

script

set -x
exe() { echo "\`\$\$  ${@/eval/} \`" ; "$@" ; }
echo
echo -----------------------------------------------------------
echo # Setup
echo Lets take a random keyframe from in.mp4:
echo
exe eval "kfn=20"
echo
echo "kf=\$(ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | grep 'frame,video,0,1' | head -$kfn | tail -1 | perl -pe 's|frame,video,0,1,.*?,(.*?),.*|\1|') "
exe eval "kf=$(ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | grep 'frame,video,0,1' | head -$kfn | tail -1 | perl -pe 's|frame,video,0,1,.*?,(.*?),.*|\1|') "
echo
echo Lets select keyframe at $kf. Here are the timestamps of the all the frames from in.mp4 around this keyframe.
echo
exe eval "ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4  | perl -pe 's|frame,video,0,(.*?),.*?,(.*?),.*|\2  \1|' | perl -pe 's|(.*?)  1|\1\tKF|' |  perl -pe 's|(.*?)  0|\1|' |grep -A 5 -B 5 --color $kf"
echo
echo Lets compare 2 methods of split: actual losslesscut 3.53.0 and another one
echo

Output

---

Lets take a random keyframe from in.mp4:

$$ kfn=20

kf=$(ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | grep 'frame,video,0,1' | head -20 | tail -1 | perl -pe 's|frame,video,0,1,.?,(.?),.*|\1|')

$$ kf=3.803792

Lets select keyframe at 3.803792. Here are the timestamps of the all the frames from in.mp4 around this keyframe.

$$ ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | perl -pe 's|frame,video,0,(.*?),.*?,(.*?),.*|\2 \1|' | perl -pe 's|(.*?) 1|\1\tKF|' | perl -pe 's|(.*?) 0|\1|' |grep -A 5 -B 5 --color 3.803792

3.720375
3.737083
3.753750
3.770417
3.787125
**3.803792**   KF
3.820500
3.837167
3.853833
3.870542
3.887208

Collar

Answer 15

If you want to log only specific commands, rather than every single line, set -x is somewhat difficult to use. But you can use this function to print any command, prefixed with $ :

function run() {
    echo -n '$'  # Print $ without a newline.
    for arg in "$@"; do
      printf " %q" "$arg"  # Print $arg, properly escaped.
    done
    echo  # Print a newline.
    "$@"
}

Usage:

run ls /full/path/to/some/dir

Using printf with the %q format specifier makes sure that the output is printed in a way suitable for copying and pasting into a shell, even in the presence of spaces or other special characters. It seems to escape spaces using \ rather than adding quotes, but the output does correctly represent what's actually being run.

Maidy

Answer 16

I use the following to control exactly what I want to see:

#!/bin/bash

cmd="ls $PWD"
echo $cmd; eval $cmd

cmd="cat $PWD/whatfille.txt"
echo "-> $cmd"; eval $cmd      #this prefixes to command for better reading

The key to expand a variable is to use double quote. Single quote would not expand.

I sometimes use "logger" instead of "echo" to send to /var/log/syslog since they are often run non-interactively.

Thermostat

Answer 17

$ cat exampleScript.sh
#!/bin/bash
name="karthik";
echo $name;

bash -x exampleScript.sh

Output is as follows:

Plea

What are the colour definitions for your terminal (RGB, numerical)? – Trilinear