How to set cleanup.policy 'delete' AND 'compact' for a Kafka topic?

Question

The description of the Kafka topic cleanup.policy configuration is

A string that is either "delete" or "compact" or both. [...]

I wonder how to set both values. I am not able to get it work. A try to change a configuration this way has not the desired effect:

c:\Progs\kafka_2.12-2.2.0\bin\windows>kafka-configs.bat --zookeeper <...> --entity-type topics --entity-name MyTopic --alter --add-config cleanup.policy=[delete, compact]
Completed Updating config for entity: topic 'MyTopic'.

c:\Progs\kafka_2.12-2.2.0\bin\windows>kafka-configs.bat --zookeeper <...> --entity-type topics --entity-name MyTopic --describe
Configs for topic 'MyTopic' are cleanup.policy=delete,segment.ms=300000,retention.ms=86400000

And this way does not work either:

c:\Progs\kafka_2.12-2.2.0\bin\windows>kafka-configs.bat --zookeeper <...> --entity-type topics --entity-name MyTopic --alter --add-config "cleanup.policy=delete compact"
Error while executing config command with args '--zookeeper <...> --entity-type topics --entity-name MyTopic --alter --add-config cleanup.policy=delete compact'
org.apache.kafka.common.config.ConfigException: Invalid value delete compact for configuration cleanup.policy: String must be one of: compact, delete
        at org.apache.kafka.common.config.ConfigDef$ValidString.ensureValid(ConfigDef.java:931)
        at org.apache.kafka.common.config.ConfigDef$ValidList.ensureValid(ConfigDef.java:907)
        at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:480)
        at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:464)
        at kafka.log.LogConfig$.validate(LogConfig.scala:305)
        at kafka.zk.AdminZkClient.validateTopicConfig(AdminZkClient.scala:319)
        at kafka.zk.AdminZkClient.changeTopicConfig(AdminZkClient.scala:331)
        at kafka.zk.AdminZkClient.changeConfigs(AdminZkClient.scala:268)
        at kafka.admin.ConfigCommand$.alterConfig(ConfigCommand.scala:152)
        at kafka.admin.ConfigCommand$.processCommandWithZk(ConfigCommand.scala:103)
        at kafka.admin.ConfigCommand$.main(ConfigCommand.scala:80)
        at kafka.admin.ConfigCommand.main(ConfigCommand.scala)

Hen

Answer 1

It should be cleanup.policy=[delete,compact] instead of cleanup.policy=[delete, compact]. No blank space follows the comma.

Manciple

I swear (fingers crossed) I tried it and while it does not raise an exception it only sets the policy to 'delete' (checked via --describe). Because of your answer I tried it again and now it worked. Thanks! 🤦‍♂️🤷‍♂️ – Hen

With blank spaces, the command only retrieves the first one and ignores the others silently. – Manciple

Answer 2

I made it work with

--add-config cleanup.policy='[compact,delete]'

Rolandorolandson

Answer 3

I'm using kafka 3.6.1. To revert cleanup policy from compact to delete, this is what I used:

kafka-configs.sh --bootstrap-server localhost:9092 --entity-type topics --entity-name my-topic --alter --delete-config cleanup.policy

Promisee