What is the Fastest way to read event log on remote machine?

Question

I am working on an application which reads eventlogs(Application) from remote machines. I am making use of EventLog class in .net and then iterating on the Log entries but this is very slow. In some cases, some machines have 40000+ log entries and it takes hours to iterate through the entries. what is the best way to accomplish this task? Are there any other classes in .net which are faster or in any other technology?

Marguerita

I don't know what is the best way to read event log items, but I'd advise you to rethink you architecture!!! You should use some other logging technology! I'd use Enterprise Library Logging Application block. Then you can easily log in a DB (you can log into the event log as well). Then you can easily run a query against the DB. – Sewage

I am not logging in event log. There are multiple applications which logs in event log and My application searches the event log based on Source and event code to get the info from it. – Marguerita

Answer 1

Man, I feel your pain. We had the exact same issue in our app.

Your solution has a branch depending on what server version you're running on and what server version your "target" machine is running on.

If you're both on Vista or Windows Server 2008, you're in luck. You should look at System.Diagnostics.Eventing.Reader.EventLogQuery and System.Diagnostics.Eventing.Reader.EventLogReader. These are new in .net 3.5.

Basically, you can build a query in XML and ship it over to run on the remote computer. Maybe you're just searching for events of a specific type, or maybe just new events from a specific point in time. The search runs on the remote machine, and then you just get back the matching events. The new classes are much faster than the old .net 2.0 way, but again, they are only supported on Vista or Windows Server 2008.

For our app when the target is NOT on Vista/Win2008, we downloaded the raw .evt file from the remote system, and then parsed the file using its binary format. There are several sources of data about the event log format for .evt files (pre-Vista), including link text and an article I recall on codeproject.com that had some c# code.

Vista and Windows Server 2008 machines use a new .evtx format that is a new format, so you can't use the same binary parsing approach across all versions. But the new EventLogQuery and EventLogReader classes are so fast that you won't have to. It's now perfectly speedy to just use the built-in classes.

Salmonoid

Answer 2

A good Explanation/Example can be found on MSDN.

EventLogSession session = new EventLogSession(Environment.MachineName);

// [System/Level=2] filters out the errors
// Where "Log" is the log you want to get data from.
EventLogQuery query = new EventLogQuery("Log", PathType.LogName, "*[System/Level=2]");
query.Session = session;

EventLogReader reader = new EventLogReader(query);

for (EventRecord eventInstance = reader.ReadEvent();
    null != eventInstance;
    eventInstance = reader.ReadEvent())
{
    // Output or save your event data here.
}

When waiting 5-20 minutes with the old code this one does it in less than 10 seconds.

Indeterminable

There's an important line missing here: query.Session = session. – Derouen

@Derouen Can you update the code, where does that go? – Empower

@Derouen Thanks, I am actually trying to port this to powershell so that was helpful. Thanks. I have it working sort of, the data isn't correct fully but it's working. – Empower

Answer 3

Event Log Reader is horribly slow... too slow. WTF Microsoft?

Use LogParser 2.2 - Search for C# and LogParser on the Internet (or you can use the log parser commands from the command line). I don't want to duplicate the work already contributed by others.

I pull the log from the remote system by having the log exported as an EVTX file. I then copy the file from the remote system. This process is really quick - even with a network that spans the planet (I had issues with having the log exported to a network resource). Once you have it local, you can do your searches and processing.

There are multiple reasons for having the EVTX - I won't get into the reasons why we do this.

The following is a working example of the code to save a copy of the log as an EVTX: (Notes: "device" is the network host name or IP. "LogName" is the name of the log desired: "System", "Security", or "Application". outputPathOnRemoteSystem is the path on the remote computer, such as "c:\temp\%hostname%.%LogName%.%YYYYMMDD_HH.MM%.evtx".)

    static public bool DumpLog(string device, string LogName, string outputPathOnRemoteSystem, out string errMessage)
    {
        bool wasExported = false;
        string errorMessage = "";
        try
        {
            System.Diagnostics.Eventing.Reader.EventLogSession els = new System.Diagnostics.Eventing.Reader.EventLogSession(device);
            els.ExportLogAndMessages(LogName, PathType.LogName, "*", outputPathOnRemoteSystem);
            wasExported = true;

        }
        catch (UnauthorizedAccessException e)
        {
            errorMessage = "Unauthorized - Access Denied: " + e.Message;
        }
        catch (EventLogNotFoundException e)
        {
            errorMessage = "Event Log Not Found: " + e.Message;
        }
        catch (EventLogException e)
        {
            errorMessage = "Export Failed: " + e.Message + ", Log: " + LogName + ", Device: " + device;
        }
        errMessage = errorMessage;
        return wasExported;
    }

Laos

Answer 4

Just had the same issue and want to share my solution. It makes a search through application, system and security eventlogs from 260 seconds (using EventLog) about a 100 times faster (using EventLogQuery). And this in a way where it is possible to check if the event message contains a pattern or any other check without the requirement of FormatDescription().

My trick is to use the same mechanism as PowerShells Get-WinEvent does and then pass it through the result check.

Here is my code to find all events within last 4 days where the event message contains a filter pattern.

string[] eventLogSources = {"Application", "System", "Security"};
var messagePattern = "*Your Message Search Pattern*";
var timeStamp = DateTime.Now.AddDays(-4);

var matchingEvents = new List<EventRecord>();

foreach (var eventLogSource in eventLogSources)
{ 
    var i = 0;
    var query = string.Format("*[System[TimeCreated[@SystemTime >= '{0}']]]",
        timeStamp.ToUniversalTime().ToString("o"));

    var elq = new EventLogQuery(eventLogSource, PathType.LogName, query);
    var elr = new EventLogReader(elq);
    EventRecord entryEventRecord;
    while ((entryEventRecord = elr.ReadEvent()) != null)
    {
        if ((entryEventRecord.Properties)
            .FirstOrDefault(x => (x.Value.ToString()).Contains(messagePattern)) != null)
        {
            matchingEvents.Add(entryEventRecord);
            i++;
        }
    }
}

Shroudlaid

Answer 5

Maybe WMI can help you:

WMI with C#

Rhoea

Answer 6

Have you tried using the remoting features in powershell 2.0? They allow you to execute cmdlets (like ones to read event logs) on remote machines and return the results (as objects, of course) to the calling session.

Sorensen

Answer 7

You could place a Program at those machines that save the log to file and sends it to your webapplication i think that would be alot faster as you can do the looping local but im not sure how to do it so i cant ive you any code :(

Misapply

Answer 8

I recently did such thing via WCF callback interface however my clients interacted with the server through WCF and adding a WCF Callback was easy in my project, full code with examples is available here

Hasidism

Answer 9

Maybe that the remote computers could do a little bit of computing. So this way your server would only deal with relevant information. It would be a kind of cluster using the remote computer to do some light filtering and the server would the the analysis part.

Mathre