Kafka topic creation: Timed out waiting for a node assignment

Question

I've got a local Kafka running using the following docker-compose.yml file:

version: '2'
services:
  zookeeper:
    image: "confluentinc/cp-zookeeper:5.0.1"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  kafka:
    image: "confluentinc/cp-enterprise-kafka:5.0.1"
    ports:
      - '9092:9092'
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 100

Trying to run a basic create topic using kafka-client 2.1.0 in Scala:

val props = new Properties()
props.setProperty(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092")

val adminClient: AdminClient = AdminClient.create(props)
val newTopic = new NewTopic("test", 1, 1.toShort)
val topicsF = adminClient.createTopics(List(newTopic).asJavaCollection)
val result = topicsF.all().get()

but after some time I get:

org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.

I can create a topic using the command line:

kafka-topics --create \
    --zookeeper localhost:2181 \
    --replication-factor 1 \
    --partitions 1 \
    --topic test
Created topic "test".

kafka AdminClient API Timed out waiting for node assignment describes a similar problem using Java but the comment suggests that a system restart fixed the issue which is not the case on my side.

Vociferous

for future reference: for those looking at this question and wondering why the environment looks the same in the question and the answer - @Vociferous edited the question after the answer, see the question history – Waldner

Answer 1

If you're running Kafka in Docker (or similar) you need to configure the listeners correctly. This article describes it in detail.

Here's an example of a Docker Compose that you can use to access Kafka from your host machine.

Disclaimer: I wrote the article :)

Alberik

I have updated my question with a docker-compose.yml file. With this setup, it still does not work. – Vociferous

where are you running your Scala code? on the Docker host or on another container? – Alberik

I start the described docker-compose.yml and run the code locally from my MacOS as I just want to quickly develop without the need to build an sbt container and deploy it. – Vociferous

I'm not familiar with the Docker image you're using. I've updated my answer with some additional material – Alberik

I have updated my question with the proposed docker-compose.yml. But when I run the simple code, it still fails with the same error. – Vociferous

Does it work if you use kafkacat as described in the article I link to? – Alberik

This link is not working: github.com/confluentinc/demo-scene/blob/master/cos/… – Beograd

@Beograd fixed: github.com/confluentinc/demo-scene/blob/master/… – Alberik

Answer 2

As @suh pointed out.

An easier approach is uncomment kafka server.properties at line :listeners=PLAINTEXT://localhost:9092 and it should work.

Dulcinea

Hi sir, the above change worked for me but may I know the reason behind the need for change ? – Vocalic

Answer 3

the main thing you have to do is

• first open your kafka server.properties from this path

  nano kafka-directory/config/server.properties

• now go to change this line. listeners=PLAINTEXT://:9092 with this

  listeners=PLAINTEXT://your ip address:9092

your ip address must be an IP of the same network with the machine where you attempt to connect to kafka i mean if the ip of the client machine is late say 192.168.10.1 your listeners ip configuration must be as:

listeners=PLAINTEXT://192.168.10.*:9092

• next modifie this configuration advertised.listeners= to this

  advertised.listeners=PLAINTEXT://your ip address:9092

thes two configurations must have the same IP and port at the end you got some thing like.

listeners=PLAINTEXT://192.168.10.10:9092
advertised.listeners=PLAINTEXT://192.168.10.10:9092

Westland

Answer 4

I think the localhost is the problem. In your bootstrap-servers properties use the advertised host (192.168.99.100) that you've defined in your compose file, instead of localhost, that should work.

Trellas

Can you look inside the container, what's the advertised host defined in the server.properties? Then you can try doing a telnet to <host> 9092 from outside, and see if it's able to connect. – Trellas

Inside of the container in server.properties: advertised.listeners=PLAINTEXT://192.168.99.100:9092. > telnet 192.168.99.100:9092 192.168.99.100:9092: nodename nor servname provided, or not known – Vociferous

I have updated my question with a docker-compose.yml file. With this setup, it still does not work. – Vociferous

For me changing "listeners=PLAINTEXT://:9092" to "listeners=PLAINTEXT://localhost:9092" in the kafka server.properties worked. Thanks! – Ploss

Answer 5

My advertised listeners were wrong and i did not have KAFKA_LISTENER_SECURITY_PROTOCOL_MAP configured.

A good example i found here:

KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:29092,PLAINTEXT_HOST://localhost:9092,SSL://kafka1:29093,SSL_HOST://localhost:9093,SASL_SSL://kafka1:29094,SASL_SSL_HOST://localhost:9094

P.S: Use only what you need, for me it was SASL_SSL & SASL_SSL_HOST.

Canopus

Answer 6

Configuring using local Kafka. un comment change and the listeners PLAINTEXT to

listeners=PLAINTEXT://127.0.0.1:9092

also, change the create topic command bootsrap-server to point to the plain text server address.

kafka-topics.bat --create  --bootstrap-server 127.0.0.1:9092 --replication-factor 1  --partitions 1 --topic TestTopic

Created topic TestTopic.

Borderline

Answer 7

for me I have changed the security in-bound rules, my adding a new rule, allow traffic from anywhere and with IP-V4 it solved my problem. but consider your security issue use only my-ip address in security in-bound rules.

Sludge