tail and grep log and mail (linux)

Question

i want to tail log file with grep and sent it via mail like:

tail -f /var/log/foo.log | grep error | mail -s subject [email protected]

how can i do this?

Disulfide

That question is like "I have print('Foo'); and I want it to print 'Foo'. How do I do that?" What exactly is your question? – Parallel

You probably don't want tail -f as this will continually follow the bottom of the file. grep doesn't tail logs, either. What is the problem that you are trying to solve? – Coalesce

tail -f .. | grep ... is to output appended data as the file grow and with use of grep, it only print append lines that match only – Tardy

Answer 1

You want to send an email when emailing errors occur? That might fail ;)

You can however try something like this:

tail -f $log |
grep --line-buffered error |
while read line
do
    echo "$line" | mail -s subject "$email"
done

Which for every line in the grep output sends an email.

Run above shell script with

nohup ./monitor.sh &

so it will keep running in the background.

Beaty

you don't need the trailing backslashes -- the shell will look to the next line for a trailing pipe (and also &, && and ||) – Unfetter

Assume I want to print out 10 lines after the error as well (grep -A 10 error). In your approach, I'll receive 10 emails, what do I need to do to receive only one mail including the 10 lines? – Taryntaryne

@Taryntaryne Did you find any approach to do what you asked – Impress

I was thinking a crontask to grep the log file using the last known line length (wc -l) and email new occurrences only. This way, the script isn't running all the time. Then, I'd have to figure out how to reset the word count whenever the log rotates. I'm surprised there aren't scripts already written to do this... many admins have this need but don't want to use a pay tool. – Electronarcosis

Only then you will have to search the whole file for those line terminators with wc and then again with tail getting the last couple of lines. – Beaty

I'm new to scripting, but it seemed like it would be less taxing to grep log every 10 minutes instead of tailing permanently. – Electronarcosis

The command in the answer is a following tail, that is; tail will poll (or monitor) the file for changes and output the new lines when it detects them. The grep will sleeping when there is no input coming. – Beaty

Answer 2

I'll have a go at this. Perhaps I'll learn something if my icky bash code gets scrutinised. There is a chance there are already a gazillion solutions to do this, but I am not going to find out, as I am sure you have trawled the depths and widths of the cyberocean. It sounds like what you want can be separated into two bits: 1) at regular intervals obtain the 'latest tail' of the file, 2) if the latest tail actually exists, send it by e-mail. For the regular intervals in 1), use cron. For obtaining the latest tail in 2), you'll have to keep track of the file size. The bash script below does that - it's a solution to 2) that can be invoked by cron. It uses the cached file size to compute the chunk of the file it needs to mail. Note that for a file myfile another file .offset.myfile is created. Also, the script does not allow path components in the file name. Rewrite, or fix it in the invocation [e.g. (cd /foo/bar && segtail.sh zut), assuming it is called segtail.sh ].

#!/usr/local/bin/bash
file=$1
size=0
offset=0
if [[ $file =~ / ]]; then
   echo "$0 does not accept path components in the file name" 2>&1
   exit 1
fi
if [[ -e .offset.$file ]]; then
   offset=$(<".offset.$file")
   fi
if [[ -e $file ]]; then
   size=$(stat -c "%s" "$file")    # this assumes GNU stat, possibly present as gstat. CHECK!
                                   # (gstat can also be Ganglias Status tool - careful).
fi
if (( $size < $offset )); then     # file might have been reduced in size
   echo "reset offset to zero" 2>&1
   offset=0
fi
echo $size > ".offset.$file"
if [[ -e $file &&  $size -gt $offset ]]; then
   tail -c +$(($offset+1)) "$file" | head -c $(($size - $offset)) | mail -s "tail $file" foo@bar
fi

Jamey

The grep part to your question was not answered in this. Left as exercise. – Jamey

Need full quotes around file name variable expansions. Path to bash is not correct for all systems; maybe invoke via env. The output of stat is highly variable between systems and versions(!). Presuming recent GNU stat, try -c to solve this and avoid the need to parse. – Ret

No cat needed: offset=$(<.offset.$file). The second part of if [[ -e $file && $size > $offset ]] is an lexical comparison so it won't do what you expect if, for example, $size == 10 and $offset == 2. Use this instead: if [[ -e $file ]] && (($size > $offset)). You can omit dollar signs in most cases inside (()) and $(()). – Rhaetia

Followed up most of the suggestions - thanks (> comparison bit me yet again, argh). Can't see my commit message (yet?), not sure why. I do not aim for total portability and don't really like env, also, I'll investigate 'most' in 'in most cases'. – Jamey

Answer 3

How about:

mail -s "catalina.out errors" [email protected] < grep ERROR catalina.out

Sama

Answer 4

I use a 1-liner in my cronjob running every 60 minutes, checking if the file has changed within the last 60 minutes. If so, the last 3 lines of the log file are sent to me.

log_file=$(find /var/log/mylog.log -mmin -60) && [[ -n "$log_file" ]] && tail -n 3 "$log_file" | mail -s "Alert mylog" [email protected]

If you want to send the last 3 lines containing the word "ERROR", like requested by the op, you can modify this as follows:

log_file=$(find /var/log/mylog.log -mmin -60) && [[ -n "$log_file" ]] && grep "ERROR" "$log_file" | tail -n 3 | mail -s "Alert mylog" [email protected]

Calysta