KafkaConsumer.close() Why?

Question

I am here asking about closing a kafka consumer. Do I need to close kafka consumer even if the using thread exits? Would not closing it leak resources by any change?

Here is a code example:

public class MyThread extends Thread{

    private KafkaConsumer<String, Message> kafkaConsumer;


    @Override
    public void run() {
        kafkaConsumer = initConsumer();
        while(true){
            kafkaconsumer.poll(1000000)
            //Code goes here.
        }
    }
}

Does the kafkaConsumer close when MyThread exits using System.exit?

Enquire

Answer 1

Quoting from "Kafka - the definitive guide" By Gwen Shapira, Neha Narkhede, Todd Palino (O' Reilly Media) :

Always close() the consumer before exiting. This will close the network connections and sockets. It will also trigger a rebalance immediately rather than wait for the group coordinator to discover that the consumer stopped sending heartbeats and is likely dead, which will take longer and therefore result in a longer period of time in which consumers can't consume messages from a subset of the partitions.

Kafka - The Definitive Guide

Perforate

It can also commit offsets: kafka.apache.org/20/javadoc/org/apache/kafka/clients/consumer/… – Hagler

Does the above text apply for kafka 1.0? – Enquire

The book was written for Kafka version 0.9.0.1 Since that date, the closing mechanism was upgraded in version 0.10.0.2 as follows "Java consumer now shuts down gracefully. By default, the consumer waits up to 30 seconds to complete pending requests. A new close API with timeout has been added to KafkaConsumer to control the maximum wait time." kafka.apache.org/10/documentation.html – Perforate