failing to load log4j2 while running fatjar

Question

i am working on a project where i utilize log4j2 logging. while developing in intellij, all works fine and the logging is done as expected. the log4j2.xml is linked through java property passed to jvm on startup via intellij settings. but once i try to run a standalone gradle built fat-jar, i'm experiencing the following problems:

java -Dlog4j.debug=true -Dlog4j.configurationFile=/home/aaa/log4j2.xml -jar /home/aaa/myjar-SNAPSHOT.jar

exceptions:

ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
...
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

i don't even understand where those [thread]s come from, since i get the same error even while using a basic simplest config in my log4j2:

<?xml version="1.0" encoding="UTF-8" ?><Configuration status="WARN" monitorInterval="86400">
<Appenders>
    <Console name="console-log" target="SYSTEM_OUT">
        <PatternLayout
                pattern="%-5p %d{yyyy-MM-dd HH:mm:ss.SSS} ${hostName} %c{1} %msg %throwable{7}%n"/>
    </Console>
</Appenders>
<Loggers>
    <Root level="info" additivity="false">
        <AppenderRef ref="console-log"/>
    </Root>
</Loggers>

any thoughts are welcome. thanks.

Showroom

ok, the problem is that log4j2-flume appender doesn't work while packed in a fatjar (errors are shown above). in a standard multi-jar classpath aware environment all works fine. any thoughts on how to solve this for fatjars? – Showroom

Anyone found a solution to this? – Faultfinding

nope, we moved to work with a main jar and lib of jars instead of a fat jar in that project. – Showroom

Answer 1

The problem is described here: https://issues.apache.org/jira/browse/LOG4J2-673

Unfortunately at the moment there only seems to be a solution for the maven-shade-plugin: https://github.com/edwgiz/maven-shaded-log4j-transformer

<plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>2.4.3</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <finalName>${project.artifactId}${appSuffix}</finalName>
                        <transformers>
...
                            <transformer
                                    implementation="com.github.edwgiz.mavenShadePlugin.log4j2CacheTransformer.PluginsCacheFileTransformer">
                            </transformer>
                        </transformers>
...
                    </configuration>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>com.github.edwgiz</groupId>
                    <artifactId>maven-shade-plugin.log4j2-cachefile-transformer</artifactId>
                    <version>2.1</version>
                </dependency>
            </dependencies>
        </plugin>
</plugins>

Griggs

To avoid using 3rd party plugins, I just used IncludeResourceTransformer to copy the right version of Log4jPlugins.dat. See https://mcmap.net/q/302354/-log4j2-dynamic-appender-doesn-39-t-work-with-maven-shade-plugin – Venison

Answer 2

in fatJar, dependencies can provide a log4j-provider.properties in the META-INF that cause this issue,

remove it in the gradle task :

task fatJar(type: Jar) {
    manifest {
        attributes 'Implementation-Title': 'project',
        'Implementation-Version': project.version,
        'Main-Class': 'com.sample.CLI'
    }
    baseName = project.name + '-all'
    from {
        configurations.compile.collect {
            it.isDirectory() ? it : zipTree(it).matching {
                exclude 'META-INF/**.RSA'
                exclude 'META-INF/MANIFEST.MF'
                exclude 'META-INF/log4j-provider.properties'
            } } }
    with jar
}

Jettiejettison

Answer 3

Kinda late to this party, but I thought I'd add my situation, in case it helps anyone.

I build my own fat jar out of a subset of classes and jars from the larger project. The essential class runs, but I was getting all the "Unrecognized format specifier" and all. Most of the answers involve mvn shade or other such, so that wasn't helping me. But poking around, I learned that log4j-web.jar also includes the Log4jPlugins.dat file, causing the problems. Removing that from the build, and all is good.

(And I thought my build script was so tricky, including all the jars by project name, eg all jars for "log4j.")

Jayson

Answer 4

The LoggerContextFactory binds the Log4j API to its implementation. The Log4j LogManager locates a LoggerContextFactory by locating all instances of META-INF/log4j-provider.properties, a standard java.util.Properties file, and then inspecting each to verify that it specifies a value for the Log4jAPIVersion property that conforms to the version required by the LogManager.

Incase of fat jar, you can also explicitly specify log4j2 to use LoggerContextFactory in your application by:

System.setProperty("log4j2.loggerContextFactory", "org.apache.logging.log4j.core.impl.Log4jContextFactory")

or as specified in the log4j-provider.properties file included in your log4j-core jar.

Acrylic

Answer 5

When you run your application from an IDE, jar runs itself without embedding the dependencies and you don't have conflict of log settings. But when you convert the application into a fat jar then all the dependencies will be injected into your project's jar file and your log4j settings that come from external jar files (dependencies) may be conflicted while fatJar process merge them into a single artifact.

In this case i think your "Log4j2Plugins.dat" files may be conflicted. To be sure, you can open your fatJar file with a zip editor (ex: 7Zip), navigate to path in fatJar as below and delete one of the conflicted files (you can choose smallest one by size) from your fatJar. Run the fatJar and check the logging is working properly.

\META-INF\org\apache\logging\log4j\core\config\plugins\Log4j2Plugins.dat

Now we can check the dependencies (artifacts) and find which of them contain the "Log4j2Plugins.dat" files. So you can exclude the modules that have the file from your build tool and then your fatJar creation process will exclude the conflicted files and your new fatJar can start logging as expected.

In my case, my fatJar module imports some other modules from Spring Boot and when i exclude the conflicted logging libraries, my fatJar starts logging without any error.

configurations { all*.exclude module: 'spring-boot' all*.exclude module: 'spring-boot-starter-logging' all*.exclude module: 'logback-classic' all*.exclude module: 'commons-logging' }

Mensurable

Answer 6

You can exclude Log4j2Plugins.dat from other libraries, and keep only the one in log4j2-core. For example in maven-shade-plugin:

<filter>
    <artifact>SOME-ARTIFACT-1</artifact>
    <excludes>
        <exclude>**/Log4j2Plugins.dat</exclude>
    </excludes>
</filter>
<filter>
    <artifact>SOME-ARTIFACT-2</artifact>
    <excludes>
        <exclude>**/Log4j2Plugins.dat</exclude>
    </excludes>
</filter>
<!-- etc -->

Another option is to copy the right file, like suggested in another answer here. Although I prefer to rely on official plugins like I did here.

The end goal is to end up with the right version of Log4j2Plugins.dat.

Venison