How to delete multiple topics in Apache Kafka

Question

Assuming that I have a number of topics with the same prefix, e.g:

giorgos-topic1
giorgos-topic2
giorgos-topic3
...

The command used for deleting a single topic (say giorgos-topic1) is the following:

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic giorgos-topic1

Is it possible to delete multiple topics using a single command and possibly a regular expression/wildcard (e.g. giorgos-*) instead of typing all the topic names that need to be deleted one by one?

Controversial

Does anyone what what should be the regex for digits only topic-names? – Decerebrate

Answer 1

Yes you can use regex-like expressions when deleting topics with the kafka-topics.sh tool:

For example, to delete all topics starting with giorgos-:

./bin/kafka-topics.sh --bootstrap-server localhost:9092 \
  --delete --topic 'giorgos-.*'

---

Using the Admin APIs, you can also delete several topics at once, see AdminClient.deleteTopics

Cloots

how about deleing multiple patterns, e.g., foo.* bar.* ? – Eastereasterday

It seems AdminClient.deleteTopics supports topic names instead of topic name patterns in its arguments. Is there a way to delete by patterns without querying in advance? – Donica

Answer 2

In cases where regex is not possible we can use a comma seperated list of topic names for the deletion of topics.

kafka-topics.sh --zookeeper 10.0.0.160:2181 --delete --topic giorgos-topic1,giorgos-topic2,giorgos-topic3,...

Teat

Regex is always possible. For this example, giorgos-topic[123]. You can use OR too. (some-topic|my-other-topic_1) – Licensee

Answer 3

Just to add to the accepted answer, the * wildcard needs to be preceded by '.'

In my experience:

--delete --topic '_confluent-controlcenter-5-3-1-1-.*' works

--delete --topic '_confluent-controlcenter-5-3-1-1-*' DOESNT work

Note: I would've added this as a comment but don't have enough rep.

Kingwood

That's because it's a regular expression, not glob syntax. Also, the accepted answer did imply you needed a .* – Swallowtailed

Answer 4

Please add single quote ('giorgos-.* ') if it complains like no matches found: giorgos-.*

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic 'giorgos-.*'

Ricoriki

Answer 5

Recent Kafka versions will remove the Zookeeper dependency and going forward you need to reference the brokers, through --boostrap-server:

kafka-topics \
    --bootstrap-server localhost:9092,localhost:9093,localhost:9094 \
    --delete \
    --topic 'giorgos-.*'

Controversial