Kafka-python retrieve the list of topics

Question

I'm using kafka-python and I'm wondering if there is a way for showing all the topics.

Something like this:

./bin/kafka-topics.sh --list --zookeeper localhost:2181

Multiphase

did you try with kafka-python.readthedocs.org/en/1.0.2/apidoc/… ? – Brewhouse

Many thanks, is what i was looking for. If you want to write an answer I will mark it as correct. Maybe could be useful also for others. – Multiphase

Answer 1

import kafka
consumer = kafka.KafkaConsumer(group_id='test', bootstrap_servers=['server'])
consumer.topics()

Denote

Sounds good, doesn't work "ValueError: Topic name "*" is illegal, it contains a character other than ASCII alphanumerics, ".", "_" and "-"" – Soucy

Answer 2

You need to use the KafkaAdminClient, not KafkaConsumer:

import kafka
admin_client = kafka.KafkaAdminClient(bootstrap_servers=['server'])
admin_client.list_topics()

Extrusion

Answer 3

Try with the method KafkaConsumer.topics().

Brewhouse

This does not seem to poll Kafka for the available topics, only lists the topics that a consumer instance is subscribed to. – Forepleasure

This is not a static method, so must create an instance of KafkaConsumer to use it, but it turns out you can do that without a topic list, and it will in fact list the topics from kafka – Forepleasure