Kafka-topics.sh authentication

Question

I am learning Apache Kafka and I do not understand how to make kafka-topics.sh work with configured SASL_PLAINTEXT authentication on the server.

This is a server.properties content:

security.protocol=SASL_PLAINTEXT
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
security.inter.broker.protocol=SASL_PLAINTEXT

listeners=SASL_PLAINTEXT://10.10.10.16:9092
advertised.listeners=SASL_PLAINTEXT://10.10.10.16:9092

listener.name.sasl_plaintext.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
   username="admin" \
   password="some-pass-1" \
   user_admin="some-pass-1" \
   user_myproducer="some-pass-2" \
   user_myconsumer="some-pass-3";

This is JAAS file content which I'm providing with KAFKA_OPTS before running kafka-topics.sh:

Client {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  security_protocol="SASL_PLAINTEXT"
  sasl_mechanism="PLAIN"
  username="admin"
  password="some-pass-1";
};

This is kafka.log content and errors I keep getting:

[2021-10-28 03:48:10,887] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.10.10.16 (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)
[2021-10-28 03:48:11,100] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.10.10.16 (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)
[2021-10-28 03:48:11,325] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.10.10.16 (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)
[2021-10-28 03:48:11,730] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.10.10.16 (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)
[2021-10-28 03:48:11,936] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.10.10.16 (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)

Any clues are appreciated

Offutt

Can you share the command you run in order to start kafka-topics.sh? – Halfbound

./kafka-topics.sh --list --bootstrap-server 10.10.10.16:9092 – Offutt

Answer 1

You can only load SASL credentials from JAAS files. The other client settings have to be provided via a configuration file. You can also provide SASL credentials via a configuration file.

For example, create a file config.properties with the following content:

sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="USER" password="PASSWORD";
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN

Then run the kafka-topics.sh tool using:

./kafka-topics.sh --list --bootstrap-server 10.10.10.16:9092 --command-config config.properties

Note that when using SASL_PLAINTEXT, your credentials will be sent over the network in clear. You should enable SSL to encrypt communications between clients and brokers.

Halfbound