Error while executing Kafka on port 2181 - topic command: Timed out waiting for a node assignment. OS Win 10

Question

I have configured kafka_2.11-2.3.0 and apache-zookeeper-3.5.5-bin on Windows 10. But while running the topic creation command I am getting the below error:

C:\kafka_2.11-2.3.0>.\bin\windows\kafka-topics.bat --create --bootstrap-server 127.0.0.1:2181 --partitions 1 --replication-factor 1 --topic testD1
Error while executing topic command : org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
[2019-10-14 16:42:40,603] ERROR java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
        at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
        at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
        at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
        at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
        at kafka.admin.TopicCommand$AdminClientTopicService.createTopic(TopicCommand.scala:178)
        at kafka.admin.TopicCommand$TopicService$class.createTopic(TopicCommand.scala:149)
        at kafka.admin.TopicCommand$AdminClientTopicService.createTopic(TopicCommand.scala:172)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:60)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.

Read somewhere in stackoverflow to add listeners=PLAINTEXT://127.0.0.1:9092 in the server.properties file but that didn't work out as expected.

Print

--bootstrap-server 127.0.01:2181 seems to be wrong -- 1) localhost address is 127.0.0.1, and 2) it should be kafka server not ZK, so the port must be 9092 (default). – Canossa

@mazaneicha: No it should be zookeeper as it manages the Kafka instance and replacing bootstrap-server with zookeeper worked. – Print

@p-s-rawat That doesnt sound right as the --bootstrap-server option was added since version 2.2.0, kafka.apache.org/documentation/#upgrade_220_notable – Canossa

Answer 1

Replacing bootstrap-server with zookeeper fixed the issue.

Print

Since Kafka 2.2.0, the preferred approach is to use --bootstrap-server (see issues.apache.org/jira/browse/KAFKA-7433) because it eliminates the need for direct connection to ZK. So this answer is not really the answer. Maybe you installed 2.3 client but your Kafka server is still < 2.2.0 ? – Canossa

I followed the quickstart guide (kafka.apache.org/quickstart) with version 2.5. It still didn't work with --bootstrap-server. It works with --zookeeper localhost:2181. What is the reason that it doesn't work with --bootstrap-server ? – Valois

I too tried the new version but still faced the same issue, but using zookeeper localhost:2181 works. – Nuptials

you're specifying port for zookeeper, that's why it didn't work. Should have specified port for Kafka-server, which in default case would have been 9092. – Gundry

Please don't use this. Zookeeper is a deprecated flag and will soon be completely removed – Vasomotor

Answer 2

Zookeeper runs on 2181, not Kafka (the bootstrap server)

By default, Kafka runs on port 9092 as below

kafka-topics --bootstrap-server 127.0.0.1:9092 --topic first_topic --create --partitions 3 --replication-factor 1

Nathan

Answer 3

I've struggled with the same issue on linux. The recommended way to create topics is still via the broker, you shouldn't need to connect directly to zookeeper.

It turned out to be that the shell scripts need a little more configuration when connecting to a TLS endpoint:

1. Copy the certs linked to by your jdk to a temporary location:

cp /usr/lib/jvm/java-11-openjdk-amd64/lib/security/cacerts /tmp/kafka.client.truststore.jks

2. Make a properties file (e.g. client.properties)

security.protocol=SSL
ssl.truststore.location=/tmp/kafka.client.truststore.jks

Then try running the script again, while passing the option --command-config with your properties file e.g.:

./kafka-topics.sh --bootstrap-server <server>:<port> --list --command-config client.properties

Note that the option is not consistent between the different scripts, for the console consumer / producer you'll need: --consumer.config and --producer.config

Lateshalatest

When I did that, the error changed and now I get WARN [AdminClient clientId=adminclient-1] Connection to node -1 (kafka-docker_kafka_1/172.18.0.2:9092) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. (org.apache.kafka.clients.NetworkClient). I'm using a docker container, BTW. – Mastersinger

@Print 's answer worked in this case of mine, but I have to run another script that doesn't work with the zookeeper. – Mastersinger

The actual question is using the wrong port. Has nothing to do with ssl – Vasomotor

Answer 4

For version 2.* you have to create the topic using zookeper with the default port 2181 as a parameter.

For the version 3.* the zookeeper is not any more a parameter, you should use --bootstrap-server using localhost or the IP adresse of the server and the default port 9092.

Documentation

Farmstead

Answer 5

Check your broker after you get this error, Kafka broker will give you the correct IP Address in the console which is running in a different terminal.

In my case I replaced 127.0.0.1:2181 with 192.168.0.21:9092 and I was able to create a new topic successfully.

Note: use bootstrap instead of zookeeper.

Fund

Why did you change the IP? – Vasomotor

Answer 6

For mac, It worked for me as follow when I use the bootstrap server with the kafka server port. Initially, it failed when I tried with a zookeeper.

bin/kafka-topics.sh --bootstrap-server localhost:9092 --topic firsttopic --create --partitions 3 --replication-factor 1

Voe

Answer 7

I faced the same issue, but everything was working fine before.

So I changed the data directory in both the properties files: zookeeper.properties and server.properties and it started working fine again. It could be because I didn't shut down the broker and zookeeper properly before.

Lilybelle

Answer 8

It is very simple:

Follow step 3 in this quickstart guide:

kafka-topics.bat --create --topic quickstart-events --bootstrap-server localhost:9092

https://kafka.apache.org/quickstart

For Windows use .bat instead of .sh

Ceylon

Answer 9

It is very simple step.

just

bin/kafka-topics.sh --create --topic thetopicname --bootstrap-server localhost:9092
This will work for v2 and above
Take note..the "thetopicname" is the name of your topic and if you are using window use
.\bin\windows\kafka-topics.bat --create --topic thetopicname
 --bootstrap-server localhost:9092

Mattoid

Answer 10

In the version of Kafka-2.8.2, I am using below command to create a topic.

Create a kafka Topic

./bin/kafka-topics.sh --create --zookeeper localhost:2181   --replication-factor 1 --partitions 1 --topic kafka-topic-test

Response:

Created topic  kafka-topic-test.

Describe topic:

bin/kafka-topics.sh --describe --topic kafka-topic-test --zookeeper localhost:2181

Respose:

Topic: kafka-topic-test TopicId: S02HD1pvSJGMrR4mIq0lmg PartitionCount: 1   ReplicationFactor: 1    Configs:
    Topic: druid-test   Partition: 0    Leader: 1001    Replicas: 1001  Isr: 1001

Muscadine

Answer 11

C:\Users\penta\kafka\bin\windows>kafka-topics.bat --create --bootstrap-server 127.0.0.1:9092 --replication-factor 1 --partitions 1 --topic test

Created topic test.

Two changes made it working:

1. removed zookeeper and replaced with --bootstrap-server
2. localhost:2181 with 127.0.0.1:9092

Sheriff

Answer 12

FOR WINDOWS the below code will work finekafka-topics.bat --bootstrap-server localhost:9092 --topic first-topic-test --create --partitions 3 --replication-factor 1

instead of zookeeper we have to use bootstrap and its port number

Barnet

Answer 13

!!! For who one is using helm chart !!!

you should change listeners config from SASL_PLAINTEXT to PLAINTEXT

Benevolent