Logback SMTPAppender Limiting Rate

Question

How can I limit the rate of emails a Logback SMTPAppender, so that it would email me at most once every n minutes?

I have setup my logging according to the Logback appender, but I don't quite see how it be configured or subclassed to implement that.

Is there a hidden feature? Did someone develop a subclass to handle this?

Potpie

Rate-limiting is one of those frustratingly tricky problems to solve. I'd be surprised if logback had a simple solution for this. – Agley

@skaffman, can you explain the complexity? I understand this may conceal some failures, in case an critical error occurs slightly after less-critial error. – Potpie

Answer 1

Based on the documentation it appears that the way to do this is to write an EventEvaluator (see example 4.14 and 4.15) which looks at the time stamp for each event to only accept an event when "enough time" has passed since the last event was accepted.

You can use System.currentTimeMillis to get a number you can do math on to calculate time differences. http://java.sun.com/javase/6/docs/api/java/lang/System.html#currentTimeMillis%28%29

Scurrile

Answer 2

As Thorbjørn, it's easy to create an EventEvaluator that limit the rate by which an appender fires a message.

However, I found Logback to support DuplicateMessageFilter, that solves my problem probably in a bitter way: "The DuplicateMessageFilter merits a separate presentation. This filter detects duplicate messages, and beyond a certain number of repetitions, drops repeated messages."

Potpie

Answer 3

Have a look at the new Whisper appender. It does smart suppression. Available via Maven and github here

Statutory disclaimer: I'm the author.

Skardol

Answer 4

To solve same problem I've written custom evaluator. It extends ch.qos.logback.classic.boolex.OnMarkerEvaluator, but you can use any other evaluator as base. If there will many acceptable messages in silence interval evaluator will discard these. For my use case it's ok, but if you need different behavior - just add extra checks to the second if.

public class LimitingOnMarkerEvaluator extends OnMarkerEvaluator {

  private long lastSend = 0, interval = 0;

  @Override
  public boolean evaluate(ILoggingEvent event) throws EvaluationException {
    if (super.evaluate(event)) {
      long now = System.currentTimeMillis();

      if (now - lastSend > interval) {
        lastSend = now;
        return true;
      }
    }

    return false;
  }


  public long getInterval() {
    return interval;
  }

  public void setInterval(long interval) {
    this.interval = interval;
  }
}

Config to send maximum one message every 1000 second (about 17 mins):

<evaluator class="package.LimitingOnMarkerEvaluator">
  <marker>FATAL</marker>
  <interval>1000000</interval>
</evaluator>

Nerine

Answer 5

This tool would do exactly what you want but it's not threadsafe at all: http://code.google.com/p/throttled-smtp-appender/wiki/Usage

I've written a threadsafe version but haven't open sourced it yet.

The reason you would have trouble finding good tools for this is that SMTP isn't a real endpoint. Use a service like loggly, airbrake, or dozens of others, or run your own server using something like logstash.

Forme

Answer 6

I suggest filing a jira item requesting this feature. It is likely to be implemented if only asked.

Ivo

Answer 7

Btw,

Logback v0.9.26 allows now to set the size of SMTPAppender message buffer. Until yesterday it would send the current contens of the buffer which was up to 256 messages which imho was a pain in the neck as I wanted to show only the last one in the email. Thus it's now possible to implement periodically recurring email warnings that carry only one particular error as per my interpretation of this question.

http://logback.qos.ch/manual/appenders.html#cyclicBufferSize

Have fun.

Eastwardly