How to configure logging for Kafka producers?

Question

I am using Kafka producer client and i don't have any log4j configuration in my project.

On running, the program prints a lot of Kafka Debug logs which i really don't want.

So, i tried to add a log4j.properties to set log level to ERROR as below which does not seem to work:

log4j.rootLogger=ERROR

How do i change Kafka Log Level?

Thrust

Answer 1

Use the command line flag -Dlog4j.configuration=file:/path/to/log4j.properties when running your client.

Example log4j property files:

• https://github.com/apache/kafka/blob/trunk/config/tools-log4j.properties
  • Used by tools like mirror maker, really just stdout
• https://github.com/apache/kafka/blob/trunk/config/log4j.properties
  • Used by tools like kafka server

For mirror maker and other tools that result in a call to kafka-run-class.sh, you can use the env variable KAFKA_LOG4J_OPTS (set to something like -Dlog4j.configuration=file:/path/to/log4j.properties) to change the logging configuration. See: https://github.com/apache/kafka/blob/0.10.2/bin/kafka-run-class.sh#L158

Example of my log4j.properties file for mirror maker that I use for testing.

# https://github.com/apache/kafka/blob/trunk/config/tools-log4j.properties

log4j.rootLogger=DEBUG, stderr

log4j.appender.stderr=org.apache.log4j.ConsoleAppender
log4j.appender.stderr.layout=org.apache.log4j.PatternLayout
log4j.appender.stderr.layout.ConversionPattern=[%d] %p %m (%c)%n
log4j.appender.stderr.Target=System.err

Steddman

Answer 2

Try adding logging.level.org.apache.kafka: DEBUG into your clients configuration properties. I am using Springboot and this is the format. Use appropriate format for your clients program.

Somnus

Cannot set level: WARNING for 'org.apache.kafka' any ideas? – Sidekick

Answer 3

I assumed you were talking about Kafka server logs. You can change log level to ERROR using following config

log4j.logger.kafka=ERROR, kafkaAppender

Hope this helps!

Dogcatcher

I am talking bout the logs that come up in my client program..not the kafka server as such... – Thrust

Answer 4

org.apache.log4j.Logger.getLogger("org").setLevel(Level.WARN);
org.apache.log4j.Logger.getLogger("akka").setLevel(Level.WARN);
org.apache.log4j.Logger.getLogger("kafka").setLevel(Level.WARN);

Drosophila

Answer 5

Spring allows us to configure the package-level logging as follows.

Steps

• Identify your logs - (I don't need to get info logs of confluent kafka packages.)

• Find out the package first - io.confluent.kafka

• Set it in the application properties

logging.level.io.confluent.kafka=WARN

In your case, you need to set the below in your application properties. I added confluent kafka as well, if you use it.

logging.level.org.apache.kafka=WARN
logging.level.io.confluent.kafka=WARN

Paschal

Answer 6

Define the logging level as below in application.yml file or your properties file.

logging:
  level:
    root: INFO
    org:
      apache:
        kafka: WARN

By defining the above, Kafka will print out only the warnings logs.

Ronaldronalda