Apache kafka 2.0.0 version - Connection to node 1 failed authentication due to: SSL handshake

Question

I'm using kafka version kafka_2.12-2.0.0 and received the below error after enabling SSL authentication. It seems to be working fine with previous versions: kafka_2.12-1.1.0, 2.11-0.10.2.2 etc.

I don't understand why it is not working with latest version 2.11-0.2.0.0? Has anyone observed the same issue that I'm facing right now with 2.0.0 version.

Below is my test environment docker config file.

listeners=PLAINTEXT://:9092,SSl://:9093
ssl.client.auth=required
ssl.keystore.location=/path/to/server.keystore
ssl.keystore.password=<Key store password>
ssl.key.password = <private key password>
ssl.truststore.location=/path/to/truststore.keystore
ssl.truststore.password=<trust store password>
security.inter.broker.protocol=SSL

And here's the error:

[2018-10-01 09:33:38,984] ERROR [Controller id=1, targetBrokerId=1] Connection to node 1 failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)

Can someone help me ?

Garibold

Please include the error you are receiving. – Orten

hi, below is the error .. [2018-10-01 09:33:38,984] ERROR [Controller id=1, targetBrokerId=1] Connection to node 1 failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient) – Garibold

Answer 1

Without more details it's hard to tell for sure, but 2.0.0 introduced a change of behaviour related to the handling of SSL connections.

As mentioned in the 2.0.0 upgrade notes, the broker setting ssl.endpoint.identification.algorithm is now set to https. This enforces hostname verification to prevent "man-in-the-middle" attacks.

To restore previous behaviour, you need to explicitely set this to an empty string.

ssl.endpoint.identification.algorithm=

Speedometer

Thank you, it helped me to fix it... now it is resolved. – Garibold

If it solved your issue, feel free to vote and accept the answer! – Speedometer

i did ssl.endpoint.identification.algorithm="" but still recieive the same error message in the log. – Sackman

It's ssl.endpoint.identification.algorithm=, no double quotes. – Endothelium

Confirmed Confluent latest docker image has the same thing as well. According to Confluent doc, docs.confluent.io/current/kafka/…, it says Kafka now verify the SSL cert hostname. It will fail if your cert sign with hostname that doesn't match the target. – Jelly

I have set ssl.endpoint.identification.algorithm= and the problem still happens. Initally, both external and inter-broker connection are failed. After i set above setting to empty string, the inter-broker connection is well but the external connection still get the output: ERROR [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient) Kafka version: 2.4.1 Can anyone give me an advice :D – Sir

Answer 2

Was also facing a similar issue. My issue, I was having Kafka server 1.1.1 running and was using Kafka client 2.1.0 to push records. Changing Kafka client to 1.1.1 solved my issue.

Hope this helps.

Thorvald