kafka bootstrap.servers as DNS A-Record with multiple IPs

Question

I have a cluster of Kafka with 5 brokers and I'm using Consul Service Discovery to put their IPs into a dns record.

kafka.service.domain.cc A 1.1.1.1 2.2.2.2 ... 5.5.5.5

Is it recommended to use only one domain name: kafka.bootstrap.servers = kafka.service.domain.cc:30000

or is it better to have multiple domain names (at least 2), each one resolves to one broker

kafka1.service.domain.cc A 1.1.1.1
kafka2.service.domain.cc A 2.2.2.2

then use them in in kafka

kafka.bootstrap.servers = kafka1.service.domain.cc:30000,kafka2.service.domain.cc:30000

my concerns with the first approach that the domain name will be resolved only once to a random broker, and if that broker is down, no new dns resolving will take place.

Agate

Answer 1

From the book Mastering Apache Kafka:

bootstrap.servers is a comma-separated list of host and port pairs that are the addresses of the Kafka brokers in a "bootstrap" Kafka cluster that a Kafka client connects to initially to bootstrap itself.

bootstrap.servers provides the initial hosts that act as the starting point for a Kafka client to discover the full set of alive servers in the cluster. Since these servers are just used for the initial connection to discover the full cluster membership (which may change dynamically), this list does not have to contain the full set of servers (you may want more than one, though, in case a server is down).

Clients (producers or consumers) make use of all servers irrespective of which servers are specified in bootstrap.servers for bootstrapping.

So as the property bootstrap.servers provides the initial hosts that act as the starting point for a Kafka client to discover the full set of alive servers in the cluster, I think both the approach will do. But as they kept the value of the property to be a comma separated list, I guess second approach will be the recommended one. And also it will be a problem in approach 1 is, while bootstrapping, random broker may be down and client will not get the cluster information to continue. So it is always better to provide more than one as fallback if one broker is down during bootstrapping.

Tavish

Answer 2

Kafka 2.1 included support for handling multiple DNS resource records in bootstrap.servers.

If you set client.dns.lookup="use_all_dns_ips" in your client configuration, it will use all of the IP addresses returned by DNS, not just the first (or a random one).

See KIP-235 and KIP-302 for more information.

Gyratory