Do a tail -F until matching a pattern

Question

I want to do a tail -F on a file until matching a pattern. I found a way using awk, but IMHO my command is not really clean. The problem is that I need to do it in only one line, because of some limitations.

tail -n +0 -F /tmp/foo | \
awk -W interactive '{if ($1 == "EOF") exit; print} END {system("echo EOF >> /tmp/foo")}'

The tail will block until EOF appears in the file. It works pretty well. The END block is mandatory because awk's exit does not exit right away. It makes awk to eval the END block before quitting. The END block hangs on a read call (because of tail), so the last thing I need to do, is to write another line in the file to force tail to exit.

Does someone know a better way to do that?

Memoirs

You can found an correct answer also on superuser.com/questions/270529/… too. – Ferryman

Somewhat related answers: superuser.com/questions/270529/… & #13984072 – Labuan

related: unix.stackexchange.com/questions/652831/… & unix.stackexchange.com/questions/607266/… – Sonorous

Answer 1

Try this:

sh -c 'tail -n +0 -f /tmp/foo | { sed "/EOF/ q" && kill $$ ;}'

The whole command-line will exit as soon as the "EOF" string is seen in /tmp/foo.

There is one side-effect: the tail process will be left running (in the background) until anything is written to /tmp/foo.

Hine

Just to explain: "sh -c" is used to run the pipe-line into a subshell, and being able to retrieve the subshell PID. The "$$" at the end of the line will be expanded to the PID of this subshell. My sed script should do the same thing as your awk script (i.e., display everything it sees, and exit when it meets the string "EOF"). Once sed has found the "EOF" string, "kill" will terminate the subshell. The "tail" dangling process will remain since it will be looping on /tmp/foo. As soon as something is written to the file, tail will echo it, and be terminated since the {...} part has exitted. – Hine

See the answer by @GregBarrett for a fix for the dangling tail process. – Claudetta

Answer 2

Use tail's --pid option and tail will stop when the shell dies. No need to add extra to the tailed file.

sh -c 'tail -n +0 --pid=$$ -f /tmp/foo | { sed "/EOF/ q" && kill $$ ;}'

Altissimo

Nice improvement! This should be the accepted answer. – Claudetta

Any way to get rid of the 7616 Terminated sh -c ... line that appears at least when I use that from a script? – Annikaanniken

If your tail does not suport the --pid option, you can use sh -i -c 'tail -n +0 -f /tmp/foo | { sed "/EOF/ q" && kill 0 ;}'. sh -i creates a new process group and kill 0 kills all processes in current process group. – Leander

Note: this does not work in BusyBox, and so it won't work on Alpine. – Balinese

@sampo you could grep -v "Terminated sh -c" it. It prints all non-matching lines. – Gelasias

Answer 3

I've not results with the solution:

sh -c 'tail -n +0 -f /tmp/foo | { sed "/EOF/ q" && kill $$ ;}'

There is some issue related with the buffer because if there aren't more lines appended to the file, then sed will not read the input. So, with a little more research i came up with this:

sed '/EOF/q' <(tail -n 0 -f /tmp/foo)

The script is in https://gist.github.com/2377029

Anthracoid

After searching through StackOverflow and the Unix Stack Exchange, this is the only answer that works for me on my Mac. I'm using it in an Xcode pre-build script. – Appalachian

Answer 4

This is something Tcl is quite good at. If the following is "tail_until.tcl",

#!/usr/bin/env tclsh

proc main {filename pattern} {
    set pipe [open "| tail -n +0 -F $filename"]
    set pid [pid $pipe]
    fileevent $pipe readable [list handler $pipe $pattern]
    vwait ::until_found
    catch {exec kill $pid}
}

proc handler {pipe pattern} {
    if {[gets $pipe line] == -1} {
        if {[eof $pipe]} {
            set ::until_found 1
        }
    } else {
        puts $line
        if {[string first $pattern $line] != -1} {
            set ::until_found 1
        }
    }
}

main {*}$argv

Then you'd do tail_until.tcl /tmp/foo EOF

Rollicking

Sweet, except I must keep it as a one-liner. – Memoirs

@shad: A one-liner without extra files? :-( – Guayule

Answer 5

Does this work for you?

tail -n +0 -F /tmp/foo | sed '/EOF/q'

I'm assuming that 'EOF' is the pattern you're looking for. The sed command quits when it finds it, which means that the tail should quit the next time it writes.

I suppose that there is an outside chance that tail would hang around if the pattern is found at about the end of the file, waiting for more output to appear in the file which will never appear. If that's really a concern, you could probably arrange to kill it - the pipeline as a whole will terminate when sed terminates (unless you're using a funny shell that decides that isn't the correct behaviour).

---

Grump about Bash

As feared, bash (on MacOS X, at least, but probably everywhere) is a shell that thinks it needs to hang around waiting for tail to finish even though sed quit. Sometimes - more often than I like - I prefer the behaviour of good old Bourne shell which wasn't so clever and therefore guessed wrong less often than Bash does. dribbler is a program which dribbles out messages one per second ('1: Hello' etc in the example), with the output going to standard output. In Bash, this command sequence hangs until I did 'echo pqr >>/tmp/foo' in a separate window.

date
{ timeout -t 2m dribbler -t -m Hello; echo EOF; } >/tmp/foo &
echo Hi
sleep 1   # Ensure /tmp/foo is created
tail -n +0 -F /tmp/foo | sed '/EOF/q'
date

Sadly, I don't immediately see an option to control this behaviour. I did find shopt lithist, but that's unrelated to this problem.

Hooray for Korn Shell

I note that when I run that script using Korn shell, it works as I'd expect - leaving a tail lurking around to be killed somehow. What works there is 'echo pqr >> /tmp/foo' after the second date command completes.

Reardon

I really like your detailed explanation. Your first command could have been the best one, except sed's behavior is the same as awk when I don't specify an END block. So I think I will keep my awk command for the moment. – Memoirs

Answer 6

Here's an extended version of Jon's solution which uses sed instead of grep so that the output of tail goes to stdout:

sed -r '/EOF/q' <( exec tail -n +0 -f /tmp/foo ); kill $! 2> /dev/null

This works because sed gets created before tail so $! holds the PID of tail

The main advantage of this over the sh -c solutions is that killing a sh seems to print something to the output such as 'Terminated' which is unwelcome

Panthea

I didn't have any luck with this in ksh. kill $! actually killed the last background job, not the tail command. – Pleasantry

Answer 7

sh -c 'tail -n +0 --pid=$$ -f /tmp/foo | { sed "/EOF/ q" && kill $$ ;}'

Here the main problem is with $$. If you run command as is, $$ is set not to sh but to the PID of the current shell where command is run.

To make kill work you need to change kill $$ to kill \$$

After that you can safely get rid of --pid=$$ passed to tail command.

Summarising, following will work just fine:

/bin/sh -c 'tail -n 0 -f /tmp/foo | { sed "/EOF/ q" && kill \$$ ;}

Optionally you can pass -n to sed to keep it quiet :)

Naldo

Answer 8

To kill the dangling tail process as well you may execute the tail command in a (Bash) process substituion context which can later be killed as if it had been a backgrounded process. (Code taken from How to read one line from 'tail -f' through a pipeline, and then terminate?).

: > /tmp/foo
grep -m 1 EOF <( exec tail -f /tmp/foo ); kill $! 2> /dev/null
echo EOF > /tmp/foo  # terminal window 2

As an alternative you could use a named pipe.

(
: > /tmp/foo
rm -f pidfifo
mkfifo pidfifo
sh -c '(tail -n +0 -f /tmp/foo & echo $! > pidfifo) | 
{ sed "/EOF/ q" && kill $(cat pidfifo) && kill $$ ;}'
)

echo EOF > /tmp/foo  # terminal window 2

Bustup

Answer 9

ready to use for tomcat =

sh -c 'tail -f --pid=$$ catalina.out | { grep -i -m 1 "Server startup in" && kill $$ ;}'

for above scenario :

sh -c 'tail -f   --pid=$$ /tmp/foo | { grep -i -m 1 EOF && kill $$ ;}'

Lithograph

That's exactly what I wanted (to check on a tomcat startup), thanks! – Tegument

Answer 10

tail -f <filename> | grep -q "<pattern>"

Aloysia

That will print nothing until the pattern, and then not quit tail. Fail. – Tombstone