ETW, .NET 4.5 - how to write to the event log?

Question

I am trying to wrap my head around ETW and how to integrate t into a high Performance application.

We all know the old dreaded EventLog with it's non structured (and thus not so optimal) API.

Now there is a new and fancy API for high performance tracing - ETW, and it got a new API on the .NET side in 4.5 in the form of the EventSource class that you can easily subclass (so no more manifest writing).

THis leaves me with a number of questions, trying to get this working.

• What is the current proposed best (as per documentation, Guidelines) way to work with ETW and get Events from there into the EventLog? I have an application here that must write (Performance) Events and would love to use ETW; but the Events shall appear in a (custom) Event log.
• Is there a complete example for this? I can find some, but they all date to the .NET 4.0 time and start with the manifest.

What did I try? I got an EventSource working, but simply have failed to get the proper documentation on how to get the rest working Downstream from there.

Urolith

This is a really good question, I have no idea on why it was downvoted. Spent a good part of a weeks evenings to read up on it - and still havnt come up with an answer. Now I have started coding and its guessing most of the way. It seems Vance and msdn is the only source of information - and Vance only promises to explain it at a later date, msdn is, well, msdn, so no help there... – Daph

Note that writing structured entries with the EventLog class is possible using the WriteEvent method msdn.microsoft.com/en-us/library/vstudio/04bh0k4k. It's not convenient, as you need to author the manifest and deploy the matching resources dll, and it uses the old eventing API, but it's possible. – Moyra

@Moyra Note that this is totally unneeded as per my comment on the anser. THe answer was right - but thoose bugs mentioned has long been fixed in a nuget release months ago. TOday you can do quite nice structured events without resolving to hacks. – Urolith

@Urolith I was just clarifying that you can do structured logging with EventLog, which you claimed isn't possible. EventSource is much better, for sure. – Moyra

Bump this question! For all the hype about ETW in C# on the few blogs, it still seems like an obscure and half baked (again) Microsoft library. I'd like to see some real world usage of ETW where events can be easily directed to 3rd party stores. – Jotting

Answer 1

What you are trying to achieve is not possible due to the following;

• To direct ETW events to the event log you’ll need to specify a channel of type Admin, Operational or some of the classic once like Application in the manifest and register it using wevtutil. Unfortunately you can’t use EventSource for this, even if you had a manifest, as the underlying implementation does not set the channel byte on the EventDescriptor block when calling WriteEvent e.g. your event is never marked for specific channel.

• What EventSource does behind the scene to avoid the cumbersome process of registering manifest, compile it into win32 resource, link it to an assembly/dll, register it etc. is to generate a manifest from your EventSource implementation and send it as a known event to allow the receiving service to parse all other event payloads instead of relying on windows infrastructure to get the manifest information. To the best of my knowledge only PerfView supports this for now.

Bluet

Thanks for this answer, Lars. I've been trying to wrap my head around ETW too, and came to the same conclusion about EventSource and channels yesterday. I didn't know about the "known event" part, but it makes sense. Unfortunately I suppose that also makes it fundamentally incompatible with event log channels, which really are the most sensible way to expose events to admins, rather than expect them to fuss with other tools, starting & stopping trace sessions, etc. :P – Peptide

Jus as final note - as per 2014 that is all wrong because microsoft did start finally changing that. RIght now a new EventSource is available at NuGet (in released form) that does add the proper channel type so that one CAN integrate them. FInally. And thanks to MS for fixing this glaring oversight ;) Just look for EventSource in Nuget and you shall find it. – Urolith

@LarsSkovslund EventSource sends the manifest as an event for events that don't go to a channel. For those which do go to a channel you need to install a manifest. The NuGet package includes tools to generate a manifest dll for you (it's actually the referenced EventRegister which does that). – Moyra

@Moyra Wrong. This is true for the one included in the .NET framework - the nuget released version can set standard channels (Admin, Operations etc.) and this works extremely nicely ;) EventSource as per Nuget download is fully usable since the new release, at least for 90% of the cases - custom channels are the issue today. works like a charm for me. – Urolith

@Urolith I am talking about the EventSource nuget package. I've never claimed it wasn't usable. You do need to install the manifests for providers that use channels, but the nuget package includes tooling that makes it easy for you. Install the EventSource nuget package (v 1.0.16 as of now) and you'll get a doc file describing the process that "ensure(s) registration artifacts (a DLL and a MAN file) will be generated in the project’s output directory for every EventSource class that needs explicit registration (currently only classes that use ETW channel support)" – Moyra

Answer 2

In August 2013, Microsoft.Diagnostics.Tracing.EventSource 1.0.4 beta released on NuGet. The three big wins are channel support, static (installed) manifest support (the two things required to get into the Event Viewer) and .NET 4.0 support.

According to the blog post announcing RTM, Microsoft.Diagnostics.Tracing.EventSource "enables fast app tracing to the Windows Event Log, including in production".

Borborygmus

Working link for blog post. – Hervey

Answer 3

This is very interesting as I have been working on the similar requirements off late. Firstly, you can generate manifest from EventSource class using its static method GenerateManifest(typeof(MyEvents), null). This will provide you with the manifest of your events etc but it does not contain details about the channels. You will need to define the channels yourself in the manifest and then register is using wevtutil.exe, mc.exe and rc.exe utilities. This will create you the event logs as per your provider name in the manifest.

Interestingly, I could manage to get the Debug and Analytic logs to show me the events generated via EventSource. I could also use Perfmon's Tracing Session to record the events for certain period with filters on keywords and levels. Provider does appear in the Trace Session provider list as well.

The only thing I am looking at present is to bring the events in Admim and Operation's channels. Please shout if you need samples etc.

Details are present at- http://www.suneet.net/FrmBlogViewer.aspx?blogid=75

Paquin

Just tried to manipulate the manifest manually to add the channel Id "chid" to the event as described below and it seems to be working. I will though test it further properly before committing to it... <event value="2001" version="0" level="win:Informational" channel="Admin" keywords="Request" opcode="Start" task="Authentication" template="AuthenticationStartArgs" message ="$(string.Event.AuthenticationStart)"/> – Paquin

Further investigation shows that we can bind events to the specific channels and it does work. However, I could not get the events in the windows event viewer if keywords are added to the events in c#. I could use trace them using Trace Session so that proves at least keywords are being attached to the event. I am yet to find the way to have keyword attached event viewable in the event viewer. – Paquin

Answer 4

The "Semantic Logging Application Block" has examples of EventListener-derived classes, one of which records in the event log. It is described a bit on Vance's blog.

Chandrachandragupta

Answer 5

With .Net 3.5 there was Microsoft.Diagnostics.Tracing.EventSource introduced, you find it on nuget and some samples here on github.

for write to etw use

using System.Diagnostics.Eventing;
    
    public void MyFunctionWithEtwLogging(){
       // ... some code
       MyEventSource.Log.Verbose("MyFunctionWithEtwLogging done");
    }

    [EventSource(Name = "MyEventSource")]
    sealed class MyEventSource : EventSource
    {
        [Event(1, Level = EventLevel.Error)]
        public void Error(string message) { WriteEvent(1, message); }

        [Event(2, Level = EventLevel.Warning)]
        public void Warning(string message) { WriteEvent(2, message); }

        [Event(3, Level = EventLevel.Informational)]
        public void Informational(string message) { WriteEvent(3, message); }

        
        public static MyEventSource Log = new LxTrackEventSource();
    }

for reading

using Microsoft.Diagnostics.Tracing.Session;
using Microsoft.Diagnostics.Tracing;

using (TraceEventSession session = new TraceEventSession(MyUniqueSessionName)) {

    var restarted = _session.EnableProvider("MyEventSource", TraceEventLevel.Verbose, ulong.MaxValue, options);
    if (restarted)  Console.WriteLine("Session restarted");

    session.Source.Dynamic.All += delegate (TraceEvent data)
    {
         Console.WriteLine("Provider: {0}" data.ProviderName)
         Console.WriteLine("TimeStamp: {0}" data.TimeStamp)
         Console.WriteLine("EventName: {0}" data.EventName)
         Console.WriteLine("Message: {0}" GetPayloadString(data));
    }
    _session.Source.Process();

}

private string GetPayloadString(TraceEvent data, int expedtedPyloadCount = 1*/Depends on the count of payload*/)
    {
    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < expedtedPyloadCount ; i++)
    {
        sb.Append(data.PayloadString(i));
    }
    return sb.ToString();
}

Quadrisect