Bash, stdout redirect of commands like scp

Question

I have a bash script with some scp commands inside. It works very well but, if I try to redirect my stdout with "./myscript.sh >log", only my explicit echos are shown in the "log" file. The scp output is missing.

if $C_SFTP; then
   scp -r $C_SFTP_USER@$C_SFTP_HOST:$C_SOURCE "$C_TMPDIR"
fi

Ok, what should I do now? Thank you

Lymphoma

Answer 1

scp is using interactive terminal in order to print that fancy progress bar. Printing that output to a file does not make sense at all, so scp detects when its output is redirected to somewhere else other than a terminal and does disable this output.

What makes sense, however, is redirect its error output into the file in case there are errors. You might want to disable standard output if you want.

There are two possible ways of doing this. First is to invoke your script with redirection of both stderr and stdout into the log file:

./myscript.sh >log 2>&1

Second, is to tell bash to do this right in your script:

#!/bin/sh

exec 2>&1

if $C_SFTP; then
   scp -r $C_SFTP_USER@$C_SFTP_HOST:$C_SOURCE "$C_TMPDIR"
fi

...

If you need to check for errors, just verify that $? is 0 after scp command is executed:

if $C_SFTP; then
   scp -r $C_SFTP_USER@$C_SFTP_HOST:$C_SOURCE "$C_TMPDIR"
   RET=$?
   if [ $RET -ne 0 ]; then
      echo SOS 2>&1
      exit $RET
   fi
fi

Another option is to do set -e in your script which tells bash script to report failure as soon as one of commands in scripts fails:

#!/bin/bash

set -e

...

Hope it helps. Good luck!

Misteach

if [ "$RET" -ne "0" ]; then.. there is no need for " around 0, -ne/-eq will test integer values. – Trstram

@Anders: Just a habit. Changed that. – Misteach

Answer 2

You cat simply test your tty with:

[ ~]#echo "hello" >/dev/tty
hello

If that works, try:

[ ~]# scp <user>@<host>:<source> /dev/tty 2>/dev/null

This has worked for me...

Mideast

I was trying to filter out the 'permanently adding <computer name> to the list of known hosts' message but wanted to show the transfer information. Redirecting to /dev/tty worked for me: scp -o StrictHostKeyChecking=no localfile remotesystem:/dir/remotefile 2>&1 >/dev/tty | grep -v "Warning: Permanently added". – Kirchner

Answer 3

Unfortunately SCP's output can't simply be redirected to stdout it seems.

I wanted to get the average transfer speed of my SCP transfer, and the only way that I could manage to do that was to send stderr and stdout to a file, and then to echo the file to stdout again.

For example:

#!/bin/sh
echo "Starting with upload test at `date`:"

scp -v -i /root/.ssh/upload_test_rsa /root/uploadtest.tar.gz speedtest@myhost:/home/speedtest/uploadtest.tar.gz > /tmp/scp.log 2>&1
grep -i bytes /tmp/scp.log
rm -f /tmp/scp.log

echo "Done with upload test at `date`."

Which would result in the following output:

Starting with upload test at Thu Sep 20 13:04:44 SAST 2012:
Transferred: sent 10191920, received 5016 bytes, in 15.7 seconds
Bytes per second: sent 650371.2, received 320.1
Done with upload test at Thu Sep 20 13:05:04 SAST 2012.

Chancelor

There's no need for a temporary file if you redirect stderr to stdout (2>&1) before piping stdout to grep, like: scp -v ... 2>&1 | grep -i bytes. – Burley

Answer 4

I found a rough solution for scp:

$ scp -qv $USER@$HOST:$SRC $DEST

According to the scp man page, -q (quiet) disables the progress meter, as well as disabling all other output. Add -v (verbose) as well, you get heaps of output... and the progress meter is still disabled! Disabling the progress meter allows you to redirect the output to a file.

If you don't need all the authentication debug output, redirect the output to stdout and grep out the bits you don't want:

$ scp -qv $USER@$HOST:$SRC $DEST 2>&1 | grep -v debug

Final output is something like this:

Executing: program /usr/bin/ssh host myhost, user (unspecified), command scp -v -f ~/file.txt
OpenSSH_6.0p1 Debian-4, OpenSSL 1.0.1e 11 Feb 2013
Warning: Permanently added 'myhost,10.0.0.1' (ECDSA) to the list of known hosts.
Authenticated to myhost ([10.0.0.1]:22).
Sending file modes: C0644 426 file.txt
Sink: C0644 426 file.txt
Transferred: sent 2744, received 2464 bytes, in 0.0 seconds
Bytes per second: sent 108772.7, received 97673.4

Plus, this can be redirected to a file:

$ scp -qv $USER@$HOST:$SRC $DEST 2>&1 | grep -v debug > scplog.txt

Morphophoneme