log4net traceappender only logs messages with level 'verbose' when using Windows Azure DiagnosticsMonitor

Question

I have an azure worker role which I have configured to use a log4net Trace Appender which writes to WindowsAzure.Diagnostics. This is done by making the following calls in the RoleEntryPoint of the worker role.

using System;
using Microsoft.WindowsAzure.Diagnostics;
using log4net.Config;

namespace XXX
{
    public class WorkerRole : RoleEntryPoint
    {
        public override bool OnStart()
        {
            var config = DiagnosticMonitor.GetDefaultInitialConfiguration();

            config.Logs.ScheduledTransferLogLevelFilter = LogLevel.Warning;
            config.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(5);

            config.WindowsEventLog.DataSources.Add("System!*");
            config.WindowsEventLog.DataSources.Add("Application!*");
            config.WindowsEventLog.ScheduledTransferLogLevelFilter = LogLevel.Error;
            config.WindowsEventLog.ScheduledTransferPeriod = TimeSpan.FromMinutes(5);

            DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", config);

        XmlConfigurator.Configure();
        }
    }
}

The App.config file is configured in the following manner:

<log4net>
    <appender name="TraceAppender" type="log4net.Appender.TraceAppender">
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%logger - %message" />
        </layout>
    </appender>
    <root>
        <level value="ALL" />
        <appender-ref ref="TraceAppender" />
    </root>
</log4net>

<system.diagnostics>
    <trace>
        <listeners>
            <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.7.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" name="AzureDiagnostics">
                <filter type="" />
            </add>
        </listeners>
    </trace>
</system.diagnostics>

The result of this is that all messages (even errors) are logged in table storage as "verbose" level.

How to fix this?

Lyrist

This is now out of date. See blog.kloud.com.au/2014/10/22/… for example. – Eller

Are you sure? People still seem to be experiencing this issue with Microsoft.WindowsAzure.Diagnostics 2.5.0.0 – Lyrist

hmmm :/ no i'm not sure. – Eller

Answer 1

A couple of blog posts deal with this issue: (here and here)

• It turns out that the log4net TraceAppender converts all log messages to Trace.Write messages and that the DiagnosticMonitorTraceListener converts all Trace.Write messages to verbose.

The answer in my case was to use Pete McEvoy's solution and extend the TraceAppender in the following manner:

using System.Diagnostics;
using log4net.Appender;
using log4net.Core;
namespace XXX.Logging
{
    public class AzureTraceAppender : TraceAppender
    {
        protected override void Append(LoggingEvent loggingEvent)
        {
            var level = loggingEvent.Level;
            var message = RenderLoggingEvent(loggingEvent);

            if (level >= Level.Error)
              Trace.TraceError(message);
            else if (level >= Level.Warn)
              Trace.TraceWarning(message);
            else if (level >= Level.Info)
              Trace.TraceInformation(message);
            else
              Trace.WriteLine(message);
            if (ImmediateFlush)
              Trace.Flush();
        }
    }
}

This extension was then implemented in my App.config:

<log4net>
    <appender name="AzureTraceAppender" type="XXX.Logging.AzureTraceAppender">
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%logger - %message" />
        </layout>
    </appender>
    <root>
        <level value="ALL" />
        <appender-ref ref="AzureTraceAppender" />
    </root>
</log4net>

Lyrist

Very useful. However, I was not getting debug level messages from log4net via Trace.Write(). I adjusted the code to else if (level >= Level.Debug) and am now seeing debug level messages in my streaming logs and table storage. – Sexual

Yes it was very useful. One comment: a simple Trace.Write(..) will not show in Azure streaming log, so the final else branch does nothing. We must project the 5 levels of log4net messages to explicitly the 3 levels of Trace. – Sheree

I have Microsoft.WindowsAzure.Diagnostics 2.5.0.0, but still the same issue. Your answer solved it though. – Immortality

as @Sheree pointed out, Trace.Write() does not show in azure streaming log. However, Trace.WriteLine() does show up as logging at the verbose level. 5 log4net levels to 4 azure levels is much easier than going to 3 azure levels – Attack

@Sheree & josh : TraceAppender code updated. Thanks for pointing it out. – Lyrist