I need to do some parsing of large (5-10 Gb)logfiles in Javascript/Node.js (I'm using Cube). The logline looks something like: 10:00:43.343423 I'm a friendly log message. There are 5 cats, and 7 do...

I'm a last year college student and I'm doing my thesis right now. My title is "Index Suggestion based on Log Analysis". This project will analyze the PostgreSQL transaction log to give i...

I have an item setup to monitor fatal errors, and I want to add another item so that it calculates the count of the fatal errors and graphs them. Using this key: log["/d2/httpd/logs/myDomain-error...

I know there are other tools around like awstats or splunk, but I wonder whether there is some serious (web)server logfile analysis going on in R. I might not be the first thought to do it in R, bu...

It's my first post so go easy on me. My problem is similar as in here, but I couldn't make it work, though I really tried: " py2exe with enthought and pandas ". Having installed separately both ve...

I have a large number (~1000) of files from a data logger that I am trying to process. If I wanted to plot the trend from a single one of these log files I could do it using plot(timevalues,data...

I have hundreds of text files with the following information in each file: *****Auto-Corelation Results****** 1 .09 -.19 .18 non-Significant *****STATISTICS FOR MANN-KENDELL TEST****** S= 609 VAR...

I am going to be undertaking some logfile analyses in R (unless I can't do it in R), and I understand that my data needs to fit in RAM (unless I use some kind of fix like an interface to a keyval s...

Can I get some recommendations (preferably with some reasons) for good log analysis software for Apache 2.2 access log files? I have heard of Webalizer and AWStats, but have never really use...

Troubleshooting, analyzing & filtering log files is by far one of the most laborious daily jobs. My issue is searching through a log file, which could be well over 4 gigs in size. Simply loadin...

I have a bunch of perfmon files that have captured information over a period of time. Whats the best tool to crunch this information? Idealy I'd like to be able to see avg stats per hour for the ob...

i am developing on 5 different rails projects, plus also refactoring some (moving from older rails versions to 2.3) - what is the best way to extract the error information from the logfiles, so i c...

I've been looking at MapReduce for a while, and it seems to be a very good way to implement fault-tolerant distributed computing. I read a lot of papers and articles on that topic, installed Hadoop...