Perl: Reading from a 'tail -f' pipe via STDIN

Question

There were a number of other threads like this, but the usual conclusion was something like "Install File::Tail". But, I'm on an old box that we're decomissioning, and I just want to write a one-liner to monitor a log. I tried installing File::Tail, but the environment for CPAN just isn't working, and I don't want to take the time to figure out what the problem is.

I just want a basic script that parses out an IP address and keeps a count of it for me. For some reason, though, even this simple test doesn't work:

$ tail -f snmplistener.log|grep IPaddress |perl -ne 'print "LINE: $_\n";'

I think it has something to do with output buffering, but I've always been a bit fuzzy on how that works. How can I get this one-liner working?

Corneous

It's not output buffering - it's input buffering. Perl won't read from stdin automatically, you'd need a while(<>) { echo $_ } type thing going. – Northeaster

False economy. If CPAN doesn't work, you can always download the distro and install it manually. – Bevin

Do you have that problem if you remove grep(1) and just perl -lne 'print "LINE: $_" if /IPaddress/' ? – Magnify

@MarcB, the -n adds a while (<>) { ... } around the provided code. – Rutaceous

@MarcB - he's thrown the -n flag. Perl will read from <> automatically :) – Magnify

Answer 1

tail -f doesn't generally buffer output, but grep probably does. Move the "grep" functionality into your Perl one-liner:

tail -f snmplistener.log | perl -ne 'print "LINE: $_\n" if /IPaddress/'

Mycobacterium

I've never seen gnu grep buffer output. tail -f|grep on an apache logfile always shows up the grepped lines immediately. – Northeaster

Oh, DUH. Ok, it works now. Marking you as answerer in 2 minutes. – Corneous

@MarcB, probably because output to a terminal is line-buffered by default, so you'd see it when tailing logs on the console. Try tail | grep | cat - or similar. Related, it's GNU grep, I believe, that supports the --line-buffered switch. :) – Magnify

Answer 2

man grep

--line-buffered
      Use line buffering on output.  This can cause a performance penalty.

so:

tail -f /log/file.txt | grep --line-buffered SomePattern | perl ...

Wersh

Answer 3

Or without using tail at all:

perl -e 'open($h,$ARGV[0]); while (1) { /IPaddress/ and print "LINE: $_" for <$h>; sleep 1 }' snmplistener.log

Sealey