tail a log file from a specific line number

Question

I know how to tail a text file with a specific number of lines,

tail -n50 /this/is/my.log

However, how do I make that line count a variable?

Let's say I have a large log file which is appended to daily by some program, all lines in the log file start with a datetime in this format:

Day Mon YY HH:MM:SS

Every day I want to output the tail of the log file but only for the previous days records. Let's say this output runs just after midnight, I'm not worried about the tail spilling over into the next day.

I just want to be able to work out how many rows to tail, based on the first occurrence of yesterdays date...

Is that possible?

Adah

Take a look to this question, there are some interesting answer that might help you: unix.stackexchange.com/questions/47407/… – Bawdyhouse

Answer 1

Answering the question of the title, for anyone who comes here that way, head and tail can both accept a code for how much of the file to exclude.

• For tail, use -n +num for the line number num to start at
• For head, use -n -num for the number of lines not to print

This is relevant to the actual question if you have remembered the number of lines from the previous time you did the command, and then used that number for tail -n +$prevlines to get the next portion of the partial log, regardless of how often the log is checked.

Answering the actual question, one way to print everything after a certain line that you can grep is to use the -A option with a ridiculous count. This may be more useful than the other answers here as you can get a number of days of results. So to get everything from yesterday and so-far today:

grep "^`date -d yesterday '+%d %b %y'`" -A1000000 log_file.txt

You can combine 2 greps to print between 2 date ranges.

Note that this relies on the date actually occurring in the log file. It has the weakness that if no events were logged on a particular day used as the range marker, then it will fail to find anything.

To resolve that you could inject dummy records for the start and end dates and sort the file before grepping. This is probably overkill, though, and the sort may be expensive, so I won't example it.

Plovdiv

Answer 2

I don't think tail has any functionality like this.

You could work out the beginning and ending line numbers using awk, but if you just want to exact those lines from the log file, the simplest way is probably to use grep combined with date to do it. Matching yesterday's date at beginning of line should work:

grep "^`date -d yesterday '+%d %b %y'`" < log_file.txt

You may need to adjust the date format to match exactly what you've got in the log file.

Ribbonfish

Answer 3

You can do it without tail, just grep rows with previous date:

cat my.log | grep "$( date -d "yesterday 13:00" '+%d %m %Y')"

And if you need line count you can add | wc -l

Disk

Answer 4

I worked this out through trial and error by getting the line numbers for the first line containing the date and the total lines, as follows:

lines=$(wc -l < myfile.log)

start=$(cat myfile.log | grep -no $datestring | head -n1 | cut -f1 -d:)

n=$((lines-start))

and then a tail, based on that:

tail -n$n myfile.log

Adah

I'm glad you worked it out however, in your answer there is and undefined variable in the code, i.e. datestring, so in start= ... grep -no $datestring ... there is not a proper context for it in the code. In other words, you should show how datestring is assigned its value too! – Nehru