Consume stdin multiple times in bash script

Question

My script accepts a stream on stdin. I want to pass the first line through to stdout no matter what, and grep the remaining lines with -v and also pass those out to stdout.

I worked out a solution using tee, but I'm wondering if this is guaranteed to always print the output of head before the output of grep? What if head was replaced with something that blocked for 20 minutes before printing anything, would that output appear at the end of stdout after the output of grep?

tee >(head -n 1) >(tail -n +2 | grep -v -E "$PATTERN")

If the order is not guaranteed, what is the right way of doing this?

Dorso

Why don't you try it? – Lientery

One more obvious problem with the two process substitutions is that although head only outputs one line, it's mostly likely reading blocks larger than one byte in order to find that first newline, and so will probably consume more than just the first line. – Osborn

Answer 1

You are overthinking this and you don't need tee, head or tail.

You can consume the first line using read and just print it out, then use grep on the rest:

$ printf "foo\nbar\nquux\n" | { read v; echo "$v"; grep -v bar; }
foo
quux

Or, combining the logic into a single awk statement and avoiding the problem altogether:

$ printf "foo\nbar\nquux\n" | awk 'NR==1{print;next} !/bar/'
foo
quux

Moor

Answer 2

You're right to be paranoid. The two sub-shells will run in parallel, so there's no guarantee that one will run before the other. To force an order of operations, read and print the first line before you grep the rest of the input.

read line && printf '%s\n' "$line"
tee >(grep -v -E "$PATTERN")

Figge

Cool yeah, that makes sense to use read in this case. I guess I was curious about the solution to this in general (where I couldn't replace one of the two with read). – Dorso

Answer 3

I think I'd go for sed:

printf "Line1\nfoo\nbar\n" | sed '1n;/bar/d'

Output:

Line1
foo

That says... if it's line 1 print and go to next, else if the line contains bar, delete it.

Boustrophedon