Parse IIS log file - is there an alternative to LogParser

Question

I need to parse an IIS log file. Is there any alternative to LogParser, a simple class to query a log file ?

I only need to know how many request I receive between 2 dates.

Here is an example of iis log file :

#Software: Microsoft Internet Information Services 7.5
#Version: 1.0
#Date: 2014-08-26 12:20:57
#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken
2014-08-26 12:20:57 W3SVC1 QXXXSXXXX 172.25.161.53 POST /XXXX/XXX/XXXX/XXXXX/1.0/XXXX/XXXXXXXX/xxxxxx.svc - 443 - 999.99.999.999 HTTP/1.1 - - - xxxx.xxxx.xxx.xxx.xxxx.xxxx.xxx.com 200 0 0 4302 5562 1560

Zoster

Can you show us the structure and sample rows of the IIS logs? PowerShell and regex looks like a good ad hoc solution here. – Sewel

Answer 1

You can use Tx (LINQ to Logs and Traces) , you can install it via nuget

and use it like this:

var iisLog = W3CEnumerable.FromFile(pathToLog);
int nbOfLogsForLastHour = iisLog.Where(x => x.dateTime > DateTime.Now.AddHours(-1)).Count();

If the log file is used by another process, you can use W3CEnumerable.FromStream

Zoster

Doesn't work, doesn't compile, does compile as framework 4.5 when set to framwork 4.0 (resets itselfs if changed), and doesn't seem to be a simple log parser. Outputs binaries to c:\bin\debug... BS. Why does a LogParser need unsafe code ? Why does a logparser need platform dependencies ? Why does it generate binaries for 4.5 when set to 4.0 ? Why does 1 logparser need to be 6 dlls ? Why does it even need Linq ? BS. Big major-leage BS. The only thing that could possibly be a worse example of BS code would be using Microsoft Log-Parser with COM-Interop. – Laurin

Answer 2

It's 2017 and the LogParser is still closed source. Moreover, all the instrumentation provided by cloud solutions appears to be making the need for parsing IIS logs a thing of the past. But since I am also dealing with legacy apps, I wrote this simple parser using .NET core.

using System;
using System.IO;
using W3CParser.Extensions;
using W3CParser.Instrumentation;
using W3CParser.Parser;

namespace W3CParser
{
    class Program
    {
        static void Main(string[] args)
        {            
            var reader = new W3CReader(File.OpenText(args.Length > 0 ? args[0] : "Data/foobar.log"));

            using (new ConsoleAutoStopWatch())
            {
                foreach (var @event in reader.Read())
                {
                    Console.WriteLine("{0} ({1}):{2}/{3} {4} (bytes sent)",
                                      @event.Status.ToString().Red().Bold(),
                                      @event.ToLocalTime(),
                                      @event.UriStem.Green(),
                                      @event.UriQuery,
                                      @event.BytesSent);
                }
            }
        }
    }
}

Source code: https://github.com/alexnolasco/32120528

Chasseur

You should make a nuget from it IMHO – Matchmark

Answer 3

You can use IISLogParser , and install it via nuget, it has support for large files (> 1Gb)

List<IISLogEvent> logs = new List<IISLogEvent>();
using (ParserEngine parser = new ParserEngine([filepath]))
{
    while (parser.MissingRecords)
    {
        logs = parser.ParseLog().ToList();
    }
}

Wagoner

Answer 4

If you're dealing with large volumes and/or dispersed locations of IIS log files, then SpectX is a handy tool for this because you don't have to ingest the logs and can run queries directly on multiple raw files. Avg processing speed per core - 350MB/sec.

It's not open source but the full-functionality 30-day trial is free.

Tutorials: Parsing IIS logs. Analyzing IIS logs - 20 sample queries.

To filter a time period, sort the logs by date and filter the period you need, e.g:

    | sort(date_time desc)
    | filter(date_time > T('2019-11-01 08:48:20.000 +0200')) 
    | filter(date_time < T('2019-11-05 11:48:20.000 +0200'));

Jordan

Answer 5

I use filter feature of CMTrace.exe tool (Refer screenshot):

Tacho