SASL authentication in docker zookeeper and kafka

Question

Can anyone please help in enabling SASL authentication with wurstmeister/zookeeper and wurstmeister/kafka in docker compose? I run these without authentication and everything works fine, but I am not able to setup simple username/password authentication.

  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"

  kafka:
    build: ./kafka
    depends_on:
      - zookeeper        
    ports:
      - "9095:9095"
    hostname: kafka
    environment:
      KAFKA_ADVERTISED_PORT: 9095 
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_HOST_NAME: kafka
      KAFKA_LISTENERS: SASL_PLAINTEXT://:9095
      KAFKA_ADVERTISED_LISTENERS: SASL_PLAINTEXT://kafka:9095
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf"
      KAFKA_INTER_BROKER_LISTENER_NAME: SASL_PLAINTEXT
      KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
      KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN      
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock  
      - ./kafka_server_jaas.conf:/etc/kafka/kafka_server_jaas.conf

kafka_server_jaas.conf

KafkaServer {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="admin"
  password="admin-secret"
  user_admin="admin-secret";
};

Client {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="admin"
  password="admin-secret"
  user_admin="admin-secret";
};

I plan to connect to Kafka inside a docker container that is running Kafkajs on Node.js

Camisado

Did you have any luck with this ? – Goodfellowship

Answer 1

I got SASL authentication working with the wurstmeister images with the below config.

docker-compose.yml:

version: '3.7'
services:
  zookeeper:
    image: wurstmeister/zookeeper:3.4.6
    environment:
      JVMFLAGS: "-Djava.security.auth.login.config=/etc/zookeeper/zookeeper_jaas.conf"
    volumes:
      - ./zookeeper_jaas.conf:/etc/zookeeper/zookeeper_jaas.conf
    ports:
     - 2181:2181
     
  kafka:
    image: wurstmeister/kafka:2.13-2.8.1
    depends_on:
      - zookeeper
    ports:
      - 9092:9092
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENERS: INTERNAL://:9093,EXTERNAL://:9092
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:9093,EXTERNAL://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
      ALLOW_PLAINTEXT_LISTENER: 'yes'
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
      KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
      KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/kafka/kafka_jaas.conf"
    volumes:
      - ./kafka_server_jaas.conf:/etc/kafka/kafka_jaas.conf

zookeeper_jaas.conf:

Server {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    user_admin="admin-secret";
};

kafka_server_jaas.conf:

KafkaServer {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="admin"
  password="admin-secret"
  user_admin="admin-secret";
};

Client {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="admin"
  password="admin-secret";
};

Needed to set the below additional kafka client properties in the producer/consumer:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";

Milkwhite

This does not working for me, getting ERROR SASL authentication failed using login context 'Client' with exception: {} (org.apache.zookeeper.client.ZooKeeperSaslClient) javax.security.sasl.SaslException: Error in authenticating with a Zookeeper Quorum member: the quorum member's saslToken is null. – Omentum

Just verified that it still works fine for me. When are you getting this error? If it is from the producer or consumer, they do need to specify the 3 client properties that I have mentioned above in the answer – Milkwhite

It happens during startup. no producer or consumer yet. – Omentum

You da man! What a nice cheat sheet. – Sliver

This does not work me too – Collyrium

@Omentum I got the same error. Have you fixed the issue? – Collyrium

Probably one of the better answers around this issue, and I figured out what the issue can be when facing the quorum member's saslToken is null. I ran into the same issue, and it had to do with fact that the LoginModule of the Client in kafka_server_jaas.conf didn't match the one in zookeeper_jaas.conf. Using the same org.apache.zookeeper.server.auth.DigestLoginModule fixed this issue, and I've confirmed this using confluentinc/cp-server and confluentinc/cp-zookeeper. Hope this helps you @Collyrium @Omentum – Homochromous