ACCEPT vs NEUTRAL in treshold filter for log4j

Question

If I use a simple Threshold filter in a log4j xml file , this simply means log everything from filter level up. So

<ThresholdFilter level="INFO" onMatch="NEUTRAL"/>

logs INFO, WARNING, ERROR and FATAL. So it seams that NEUTRAL, which is the default value, means: don't filter out anything and let the log go through.

If I change this to

<ThresholdFilter level="INFO" onMatch="ACCEPT"/>

what is it supposed to do?

Emmerie

Answer 1

First I just want to mention one item before I answer your question. You wrote:

logs INFO, WARNING, ERROR and FATAL. So it seams that NEUTRAL, which is the default value, means: don't filter out anything and let the log go through.

That's not entirely accurate in that TRACE and DEBUG level events are not published because they are filtered out by the default value of the onMismatch parameter of your filter. According to the ThresholdFilter documentation:

onMismatch [is the] Action to take when the filter does not match. May be ACCEPT, DENY or NEUTRAL. The default value is DENY.

Getting back to your question, the difference between NEUTRAL and ACCEPT when using a single filter is nothing, they behave the same. When you use a CompositeFilter, however, there's a huge difference.

Let's say I have a simple class that generates some logs:

package example;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.Marker;
import org.apache.logging.log4j.MarkerManager;

public class SomeClass {

    private static final Logger log = LogManager.getLogger();
    private static final Marker FLOW_MARKER = MarkerManager.getMarker("FLOW");

    public static void main(String[] args){

        if(log.isDebugEnabled())
            log.debug(FLOW_MARKER,"This is some debug!");

        log.info("Here's some info!");
        log.error("Some erorr happened!");

        log.trace("Trace message, yeah!");
    }
}

Now let's say I have a log4j2 configuration that includes a CompositeFilter that contains 2 filters:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
            <Filters>
                <ThresholdFilter level="INFO" onMatch="NEUTRAL" onMismatch="NEUTRAL"/>
                <MarkerFilter marker="FLOW" onMatch="ACCEPT" onMismatch="DENY"/>
            </Filters>
        </Console>

    </Appenders>

    <Loggers>
        <Root level="trace">
            <AppenderRef ref="Console" />
        </Root>
    </Loggers>
</Configuration>

When we run this simple class we get the following output:

22:35:31.666 [main] DEBUG example.SomeClass - This is some debug!

If we now change the ThresholdFilter's onMatch parameter to ACCEPT:

<ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="NEUTRAL"/>

We will get this output instead:

22:36:00.939 [main] DEBUG example.SomeClass - This is some debug!
22:36:00.941 [main] INFO  example.SomeClass - Here's some info!
22:36:00.941 [main] ERROR example.SomeClass - Some erorr happened!

At first, since the ThresholdFilter did not accept or deny any log events they all pass through to the MarkerFilter and are denied if they do not have the FLOW marker. When we changed the ThresholdFilter to accept matching events the INFO and ERROR level events were accepted before reaching the MarkerFilter and therefore were published. We still see the log message from the DEBUG level event that was not denied by the ThresholdFilter because ThresholdFilter is NEUTRAL to mismatches and MarkerFilter accepted the event because it had the FLOW marker.

Headwork

Answer 2

Answering your question in short: Effectively, nothing changes when you change your code from "NEUTRAL" to "ACCEPT". But this is only true if you have just one single filter (as D.B. also mentioned).

"NEUTRAL" and "ACCEPT" only makes a difference when you use several filters.

If you like thinking in sets:
You can devide all logs into two sets regarding one single filter:
* either the log-message (/event) is matched by this filter
* or the log-message (/event) is not matched by this filter

If you always use "NEUTRAL"-"DENY" combinations (onMatch="NEUTRAL"+onMismatch="DENY" or onMatch="DENY"+onMismatch="NEUTRAL") then you get an intersection of all "NEUTRAL"-subsets regarding to the filters. With "NEUTRAL"-"ACCEPT" combinations you get union sets.

But you need to be aware of the following rule: As soon as a filter triggers an "ACCEPT" or a "DENY" then no other filters will be evaluated. So the order of your filters is important. This is especially true for composite filters (the <filters>-tag).

And you can have filters in different locations. I think, but I'm still not absolutely sure, that the order is evaluated in the following order:

1. Context-wide Filters are configured directly in the configuration. Events that are rejected by these filters will not be passed to loggers for further processing. Once an event has been accepted by a Context-wide filter it will not be evaluated by any other Context-wide Filters nor will the Logger's Level be used to filter the event. The event will be evaluated by Logger and Appender Filters however.
2. Logger Filters are configured on a specified Logger. These are evaluated after the Context-wide Filters and the Log Level for the Logger. Events that are rejected by these filters will be discarded and the event will not be passed to a parent Logger regardless of the additivity setting.
3. Appender Filters are used to determine if a specific Appender should handle the formatting and publication of the event.
4. Appender Reference Filters are used to determine if a Logger should route the event to an appender.

Source: https://logging.apache.org/log4j/2.0/manual/filters.html

Weevily

So the order of your filters is important. , you are saying, makes one wonder what if you use the java.lang.Properties Log4j2 configuration file format (which in memory is basically an unordered (key,value) Map) how would you impose ordering of your filters? – Conformance

Answer 3

When describing Filters I think in terms of iptables firewall rules, if you are familiar with them. When a filter is configured with an onMatch value of ACCEPT it means "accept the log event regardless of other factors". When the onMatch value is set to DENY it means "reject the log event regardless of other factors". When the value is set to NEUTRAL it simply means it wasn't rejected or accepted and further evaluation of the log event should continue.

As @amin.miedl notes there are 4 places a filter can be placed:

1. Globally - These are configured outside of the appenders and loggers. They are processed first even before the LogEvent is routed to a Logger.
2. On a Logger - A LoggerConfig is selected for the LogEvent and any filters configured on it are checkeed.
3. On an Appender reference - After passing through the Logger config the Appender References are chosen. Before passing the event to the Appender any filters configured in the Appender reference are evaluated.
4. On an Appender. The Appender is the last stop for processing a Logging Event. Before the Appender receives it a wrapper class for the Appender, called a LoggerControl, calls any Filters configured on the Appender.

It should also be noted that Loggers and Appender references can be configured with a Logging Level. You should think of these as being somewhat equivalent to the ThresholdFilter being configured on the Logger and Appender Reference with an onMatch of NEUTRAL and an onMismatch value of DENY and highly optimized since virtually every logging event's logging level is evaluated.

Trestle