Can we disable log4j logs only for kafka

Question

I'm using following properties for Log4j:

//log4j.properties

log4j.rootLogger=DEBUG, stdout

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

I want to disable log messages only for Kafka, where as display my log messages being logged.

Hornbook

Answer 1

you need to disable logger for both log4j and slf4j to disable kafka logging completely:

Add a logback.xml file in your resources dir:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/base.xml" />
    <logger name="org.springframework" level="OFF"/>
    <logger name="org.apache" level="OFF"/>
    <logger name="kafka" level="OFF"/>
</configuration>

Add below to your application.yaml / properties:

logging:
  level:
    root: OFF
    org.springframework: OFF
    org.apache: OFF
    kafka: OFF

Alicea

Answer 2

You need to set the log level to OFF by adding this line:

log4j.logger.org.apache.kafka=OFF

Compare: How to disable loggers of a class or of whole package?

Statesmanship

Answer 3

A more detailed version from answer above

*. overall::

1.
Setup a simple Kafka Program (Completely from scratch)

2.
Add log4j dependency
(using Spring, so no version)

        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
        </dependency>

3.
Add the logback.xml
(same as answer above; seems like can comment out some not needed)

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/base.xml" />
<!--     <logger name="org.springframework" level="WARN"/> -->
    <logger name="org.apache" level="WARN"/>
<!--     <logger name="kafka" level="WARN"/> -->
</configuration>

*. note::

1. Add log4j dependency is needed

2. tried adding application.properties / log4j.properties in src/main/resources, seems not playing any effect

   so, config like these didnt work for me::

   #log4j.rootLogger=INFO, stdout
   #log4j.appender.stdout=org.apache.log4j.ConsoleAppender
   #log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
   #log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c)%n
   #
   #log4j.logger.kafka=OFF
   #log4j.logger.org.apache.kafka=OFF
   
   #logging.level.root=OFF
   #logging.level.org.springframework=OFF
   #logging.level.org.apache=OFF
   #logging.level.kafka=OFF
   

   placing them in a diff folder location neither worked (eg: direct under project folder / inside src/main/java)

3. I didnt find a dynamic way to add the config. (written inside programming code, instead of a file; neither in kafka config, nor in log4j getLogger)

4. place the file under src/main/resources

5. (though I am using Spring framework, I didnt use the integrated Kafka, & didnt ran it as Spring server -- just more like a normal Java application.)

6. (I dont know if there would be any confliction on the log4j (seems happened to me long before))

Oleum

Answer 4

In our case, we are using ReactiveKafkaProducerTemplate to publish records to Kafka. And extra logs were being written by TracingProducerInterceptor which was registered in the bean of ReactiveKafkaProducerTemplate

 @Bean
  public ReactiveKafkaProducerTemplate<YourModelEventKey, YourModelEvent> reactiveKafkaProducerTemplate(
      final KafkaProperties properties) {
    var props = properties.buildProducerProperties();
    props.put(ProducerConfig.INTERCEPTOR_CLASSES_CONFIG, Collections.singletonList(TracingProducerInterceptor.class));
    return new ReactiveKafkaProducerTemplate<>(SenderOptions.create(props));
  }

Solution: logging.level.brave.Tracer=warn for application.properties and for application.yaml use below

logging.level
    brave.Tracer: warn

Note: we are interested only in warnings or errors. If you want, you can fully disable by setting OFF

Cruck

Answer 5

logging:
  level:
    root: OFF
    org.springframework: OFF
    org.apache: OFF
    kafka: OFF

This configuration works for me.

Shirelyshirey

This is exactly the same as the second part of Dean Jain's answer. – Sobel