Different log files for multiple threads using log4j2

Question

I am running a Java application in which I am invoking multiple threads, each with some unique names. Now I want to create multiple log files for each of them and the name of the log files should be as the thread names. Is this possible using log4j2. Please help me write log4j2 configuration files.

Thank you in advance.

Basidium

Answer 1

This can be done with the RoutingAppender. The FAQ page has a good example config.

Marozik

I think RoutingAppender is suitable where we have some loggers previously defined and dynamically we have to choose the logger to which we want to dump the logs. But in my situation I have to create multiple loggers for each threads and the log file name will be based on the thread invocation time & the process ID or Thread name etc. Please help me with this. If this can be done using RoutingAppender then give some hints to proceed with this. Thank you – Basidium

To be honest your requirements sound very complex. Is there not an easier way? But assuming you cannot control the design, I still think you can do what you describe with RoutingAppender. Please take another look at the RoutingAppender example in the FAQ. The last Route dynamically creates a log file with a name based on the ThreadContext value. The ThreadContext map has thread-local values so each thread can put some value based on the aspects you mentioned (time, pid, tid). This value then becomes part of the file name so you get a separate log file for that. – Marozik

Answer 2

I agree a RoutingAppender is the way to go. I initially used the routing appender in conjunction with the ${ctx:threadName} lookup where the 'ctx' uses the ThreadContext. I found that I would have to sprinkle in the code a line like this:

ThreadContext.put("threadName", Thread.currentThread().getName());

While that code works it's not extensible in the design of the code. If I were to add a new java.lang.Runnable to the code base, I would have to also include that line.

Rather, the solution seems to be to implement the 'org.apache.logging.log4j.core.lookup.StrLookup' and register the @Plugin with the PluginManager Like this:

Class: ThreadLookup

package my.logging.package    
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.lookup.StrLookup;

@Plugin(name = "thread", category = StrLookup.CATEGORY)
public class ThreadLookup implements StrLookup {

@Override
public String lookup(String key) {
    return Thread.currentThread().getName();
}

@Override
public String lookup(LogEvent event, String key) {
    return event.getThreadName() == null ? Thread.currentThread().getName()
            : event.getThreadName();
}

}    

Configuration: log4j2.xml (packages attribute of the Configuration registers the @Plugin with the PluginManager)

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" packages="my.logging.package">
    <Appenders>
        <Routing name="Routing">
            <Routes pattern="$${thread:threadName}">
                <Route>
                    <RollingFile name="logFile-${thread:threadName}"
                    fileName="logs/concurrent-${thread:threadName}.log" filePattern="logs/concurrent-${thread:threadName}-%d{MM-dd-yyyy}-%i.log">
                    <PatternLayout pattern="%d %-5p [%t] %C{2} - %m%n" />
                    <Policies>
                        <SizeBasedTriggeringPolicy size="50 MB" />
                    </Policies>
                    <DefaultRolloverStrategy max="100" />
                </RollingFile>
            </Route>
        </Routes>
    </Routing>
    <Async name="async" bufferSize="1000" includeLocation="true">
        <AppenderRef ref="Routing" />
    </Async>
</Appenders>
<Loggers>
    <Root level="info">
        <AppenderRef ref="async" />
    </Root>
</Loggers>

Scurf

Answer 3

This question and the answers was a good starting point to me, however if you are using slf4j inside your implementation and log4j only for the logging side, this could be a bit trickier, which I wanted to share here. First of all the previously mentioned log4j FAQ with the example could be found for example here: https://logging.apache.org/log4j/2.x/faq.html#separate_log_files Which was unfortunately not useful to me as we are using MDC inside slf4j instead of ThreadContext. However it was not clear to me at first sight, but you can use ThreadContext and MDC in log4j xml the same. For example:

Routes pattern="$${ctx:macska}">

could refer an

 MDC.put("macska", "cica" + Thread.currentThread().getId());

in the code. So as the same as ThreadContext.

But finally my solution was not using the MDC, as I found out log4j has EventLookup: https://logging.apache.org/log4j/2.x/manual/lookups.html#EventLookup and I modified only my log4j xml with this:

<Routes pattern="$${event:ThreadName}">

Systematist