With bash, how can I pipe standard error into another process?

Question

It's well known how to pipe the standard ouput of a process into another processes standard input:

proc1 | proc2

But what if I want to send the standard error of proc1 to proc2 and leave the standard output going to its current location? You would think bash would have a command along the lines of:

proc1 2| proc2

But, alas, no. Is there any way to do this?

Vltava

You can do such a simple redirection in rc, which is another shell. Eg: proc1 |[2] proc2. Isn't it nice? Not in bash though. – Pantsuit

Related: Piping both stdout and stderr in bash?. And here is the simplest answer to pipe BOTH stdout and stderr. – Benildis

Answer 1

There is also process substitution. Which makes a process substitute for a file.
You can send stderr to a file as follows:

process1 2> file

But you can substitute a process for the file as follows:

process1 2> >(process2)

Here is a concrete example that sends stderr to both the screen and appends to a logfile

sh myscript 2> >(tee -a errlog)

Biosphere

I tried this. It didn't work ( weston --help 2> >(less) ), and it broke my shell, I had to exit and log back in. – Pantsuit

@Pantsuit if both weston --help and less are expecting to have keyboard interaction but only 1 of them receives it, then you may be in an awkward situation. Try doing testing with something like grep instead. Plus you might find that both mouse/keyboard inputs are going to the 2nd command anyway rather than to weston. – Shinbone

If you want to redirect both stderr and stdout use |&, I learnt it from here – Batchelor

Answer 2

You can use the following trick to swap stdout and stderr. Then you just use the regular pipe functionality.

( proc1 3>&1 1>&2- 2>&3- ) | proc2

Provided stdout and stderr both pointed to the same place at the start, this will give you what you need.

What the x>&y bit does is to change file handle x so it now sends its data to wherever file handle y currently points. For our specific case:

• 3>&1 creates a new handle 3 which will output to the current handle 1 (original stdout), just to save it somewhere for the final bullet point below.
• 1>&2 modifies handle 1 (stdout) to output to the current handle 2 (original stderr).
• 2>&3- modifies handle 2 (stderr) to output to the current handle 3 (original stdout) then closes handle 3 (via the - at the end).

It's effectively the swap command you see in sorting algorithms:

temp   = value1;
value1 = value2;
value2 = temp;

Vltava

What is the value of using 1>&2- here rather than just 1>&2? I don't understand why we'd want to close fd 2, if we're just going to reopen/reassign it immediately. – Cruces

@dubiousjim, no advantage in that particular case, I suspect I did it just to be consistent - closing file handle 3 is a good idea to free it up. – Vltava

trying to get gcc's make (which is colorized on my system) to work with this "( make 3>&1 1>&2- 2>&3- ) | less -R" whereas "( ls -al 3>&1 1>&2- 2>&3- ) | less -R" works as expected. – Fabrienne

It seems like your explanations are back to front for the second two redirections. 1>&2- sets file handle 2 (original stderr) to handle 1 (original stdout) 2>&3- sets file handle 3 (copied stdout) to handle 2 (original stderr). Please correct me if I'm wrong though. btw, I would guess that the dash on 2 is to prevent new stderr data from getting sent to this buffer while it is being populated with the data from stdout. – Aegir

@aghsmith, that's not the case, the handle to the left is always the one being set, the thing on the right is being used to set it (whether handle or file name). – Vltava

@paxdiablo: ah, I think I used confusing language, though I still think the language in your main answer is a bit confusing. I had understood that value on the right was being copied over to the value on the left, which is indicated in the first line you wrote "3>&1 creates a new file handle 3 which is set to teh current 1...". but the language for second and third line eg: "1>&2- sets stdout to go to the current file handle 2" implies a push of information from left to right. – Aegir

@aghsmith: okay, let me see if I can clean it up a bit. – Vltava

Answer 3

Bash 4 has this feature:

If `|&' is used, the standard error of command1 is connected to command2's standard input through the pipe; it is shorthand for 2>&1 |. This implicit redirection of the standard error is performed after any redirections specified by the command.

zsh also has this feature.

--

With other/older shells, just enter this explicitly as

FirstCommand 2>&1 | OtherCommand

Mancini

From reading the docs, that does both standard error and output as opposed to just stderr, but it's nice to know. Time to start looking at bash 4, I think. – Vltava

The current bash manual reads "If |& is used, command's standard error, in addition to its standard output, is connected to command2's standard input". This is explicitly not what the OP wants. – Runion

@PeterA.Schneider: The OP says "leave the standard output going to its current location" which may be ambiguous. – Mancini

I'm failing to see any ambiguity. Your suggestion (1) conflates the two streams. (2) OtherCommand writes the combined data somewhere, possibly somewhere else. So it's not the same data, and it is potentially going somewhere else. That is about the opposite of the OP's wish, isn't it? – Runion

@PeterA.Schneider: Where else is standard output's current location? If proc1 outputs to stdout and to stderr and you want stderr to go to the stdin of proc2 (which is where proc1's stdout is going), then my answer accomplishes that. I gave the OP what he asked for, perhaps not what he meant to ask for. Therein lies the potential ambiguity. The OP accepted the answer which swaps stdout and stderr which is not what he asked for. – Mancini

OtherCommand may not even write to stdout! It may write the combined data to /dev/null. That is not "leaving it alone". In the accepted answer, by contrast, the original stdout's data is still available -- separate from the original stderr's data, and unchanged! -- on stderr (because proc2 doesn't process the new stderr, containing the original stdout data), so the OP can now handle it any way he wants to. I think that combining the two streams is why your solution fails the spec, in any conceivable interpretation. – Runion

@PeterA.Schneider: No one said anything at all about what OtherCommand (or the OP's proc2) does except the implication that it accepts input on its stdin. and what the second process does is not specified in the question and is irrelevant. I think you fail to see that the OP originally asked for something different than what he really wanted. I answered the question as asked. Continuing this argument is pointless. – Mancini

You are right: What the second program does is irrelevant in the OP's scenario with respect to the first program's stdout. That is so because the second program does not see, let alone alter the stdout of the first program (it does see and process, in an unknown fashion, stderr). It was the OP's request that the first program's (and hence pipeline's) stdout stay unaltered. Now in your scenario the second program does suddenly process the first program's stdout (together with its stderr). So in your scenario what the second program does with its input suddenly is relevant. – Runion

Consider proc1: echo "valuable output"; echo "diagnostic noise" >&2 and proc2: while read; do true; done. Now what 's the contents of file after (proc1 2> >(proc2)) > file vs. your suggestion (proc1 |& proc2) > file? I think in the second case it is empty, because that pattern does not "leave the standard output going to its current location". – Runion

@PeterA.Schneider: You're talking about what proc2 does with its input after it receives it. This is not at all relevant in the OP's question. Also, the subshells in your examples are unnecessary. – Mancini

I tried very patiently to explain why it becomes relevant in your solution although it wasn't originally. Do you agree that the output in the two examples is different? (I think it is, but as always I may be wrong.) In particular, in your solution there is nothing on stdout (and hence in file) any longer, although the OP asked to leave stdout intact. – Runion

Answer 4

Swapping is great as it solves the problem. Just in case you do not even need the original stdout, you can do it this way:

proc1 2>&1 1>/dev/null | proc2

The order is vital; you would not want:

proc1 >/dev/null 2>&1 | proc1

As this will redirect everything to /dev/null!

Gerek

Answer 5

None of these really worked very well. The best way I found do do what you wanted is:

(command < input > output) 2>&1 | less

This only works for cases where command does not need keyboard input. eg:

(gzip -d < file.gz > file) 2>&1 | less

would put gzip errors into less

Toccaratoccata

Answer 6

((echo one out; echo -e 'two\nerr' >&2) |wc) 2> >(wc)

Room