extracting error information from rails log files

Question

i am developing on 5 different rails projects, plus also refactoring some (moving from older rails versions to 2.3) - what is the best way to extract the error information from the logfiles, so i can see all the depreciation warnings, runtime errors and so on, so i can work on improving the codebase?

are there any services or libraries out there you can recommend, that actually help with rails logfile parsing?

Wite

Answer 1

Read about grep linux command.

http://www.cyberciti.biz/faq/howto-use-grep-command-in-linux-unix/

I don't know what error log format is in Rails, but I guest every row with warning or error contain "warning" or "error" word.

Then it would be like this:

grep -E "error|warning" logfile.txt

for bot error and warnings

grep "error" logfile.txt

for errors

grep "warning" logfile.txt

for warnings

and if you want to see new errors and warnings in real time, try this

tail -f logfile.txt | grep -E "error|warning"

tail -f logfile.txt | grep "error"

tail -f logfile.txt | grep "warning"

Hope I could help you ;) and I hope that I'm not wrong about logs format in Rails

Bochum

ok, i just tried an inital regex on how to parse rails log files which seems to work: (Processing.*?)(\n\n\n|\z) - this is seperating the requests into processable pieces... – Wite

You really want a -A (after context) or -C (both before and after context) line includer in your grep statements so you get the trace as well. For example: grep -C7 -E"error|warning" logfile.txt – Springspringboard

@Springspringboard +1 for that. The error line without context is useless :) – Scission

Answer 2

I've found the request-log-analyzer project to be very useful.

You can certain grep the log to find errors and dump them out, but this does an excellent job of gathering all the information about the different actions and how long they take.

Here's some sample output.

This is the first thing I run when I get a call saying "my site is slow and I need help fixing it."

Hoptoad and/or Exceptional are great for ongoing errors, but they don't track log running requests. Something like New Relic is good for that.

Loritalorn

great! thanks a lot for this. after searching for request-log-analyzer i also came up with more rails logging: whynotwiki.com/Rails_/_Performance jacqueschirag.wordpress.com/2008/10/15/rails-log-analyzers – Wite

Answer 3

I use hoptoadapp, http://www.hoptoadapp.com/pages/home there is a free flavor, it logs your error messages to their database, and they provide an easy to use interface. All you have to do is install this plugin: http://github.com/thoughtbot/hoptoad_notifier.

It won't help for past errors, but it is great for isolating problems with a currently running app.

Effectually

thanks for the pointer, this looks like a really handy web application. i will take a look into that. – Wite