How do I prepend to a stream in Bash?

Question

Suppose I have the following command in bash:

one | two

one runs for a long time producing a stream of output and two performs a quick operation on each line of that stream, but two doesn't work at all unless the first value it reads tells it how many values to read per line. one does not output that value, but I know what it is in advance (let's say it's 15). I want to send a 15\n through the pipe before the output of one. I do not want to modify one or two.

My first thought was to do:

echo "$(echo 15; one)" | two

That gives me the correct output, but it doesn't stream through the pipe at all until the command one finishes. I want the output to start streaming right away through the pipe, since it takes a long time to execute (months).

I also tried:

echo 15; one | two

Which, of course, outputs 15, but doesn't send it through the pipe to two.

Is there a way in bash to pass '15\n' through the pipe and then start streaming the output of one through the same pipe?

Crites

Have you tried grouping? { echo 15; one; } | two – Atalya

Answer 1

You just need the shell grouping construct:

{ echo 15; one; } | two

The spaces around the braces and the trailing semicolon are required.

To test:

one() { sleep 5; echo done; }
two() { while read line; do date "+%T - $line"; done; }
{ printf "%s\n" 1 2 3; one; } | two

16:29:53 - 1
16:29:53 - 2
16:29:53 - 3
16:29:58 - done

Anele

I +1 you just because you mentioned The spaces around the braces and the trailing semicolon are required (which would be the purpose of another question) and I didn't. :) – Atalya

Answer 2

Use command grouping:

{ echo 15; one; } | two

Done!

Atalya

Answer 3

You could do this with sed:

Example 'one' script, emits one line per second to show it's line buffered and running.

#!/bin/bash
while [ 1 ]; do
    echo "TICK $(date)"
    sleep 1
done

Then pipe that through this sed command, note that for your specific example 'ArbitraryText' will be the number of fields. I used ArbitraryText so that it's obvious that this is the inserted text. On OSX, -l is unbuffered with GNU Sed I believe it's -u

 $ ./one | sed -l '1i\
> ArbitraryText
> '

What this does is it instructs sed to insert one line before processing the rest of your file, everything else will pass through untouched.

The end result is processed line-by-line without chunk buffering (or, waiting for the input script to finish)

ArbitraryText
TICK Fri Jun 28 13:26:56 PDT 2013
...etc

You should be able to then pipe that into 'two' as you would normally.

Rachellrachelle