Exception while trying to fetch data from zookeeper node: (KeeperErrorCode: ConnectionLoss )

Question

I am using following code to extract Kafka broker list from zookeeper:

private static String getBrokerList() {
        try {
            ZooKeeper zookeeper = new ZooKeeper(zookeeperConnect, 15000, null);
            List<String> ids = zookeeper.getChildren(ZkUtils.BrokerIdsPath(), false);
            List<String> brokerList = new ArrayList<>();
            for (String id : ids) {
                String brokerInfo = new String(zookeeper.getData(ZkUtils.BrokerIdsPath() + '/' + id, false, null), Charset.forName("UTF-8"));
                JsonObject jsonElement = new JsonParser().parse(brokerInfo).getAsJsonObject();
                String host = jsonElement.get("host").getAsString();
                brokerList.add(host + ':' + jsonElement.get("port").toString());
            }
            return Joiner.on(",").join(brokerList);
        } catch (KeeperException | InterruptedException e) {
            Throwables.propagate(e);
        }
        return "";
    }

Above code is working fine when one thread executing the code at a time. However, when several threads are executing the above code it fails with the following exception occasionally:

Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /brokers/ids
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
    at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
    at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1560)

What am I doing wrong here? My zookeeper version is 3.4.6-1569965.

Collapse

Answer 1

from http://zookeeper.apache.org/doc/r3.4.9/api/org/apache/zookeeper/ZooKeeper.html#ZooKeeper(java.lang.String,%20int,%20org.apache.zookeeper.Watcher)

"Session establishment is asynchronous. This constructor will initiate connection to the server and return immediately - potentially (usually) before the session is fully established. The watcher argument specifies the watcher that will be notified of any changes in state. This notification can come at any point before or after the constructor call has returned."

You have to wait fro zookeeper connection to fully estabilish: https://www.tutorialspoint.com/zookeeper/zookeeper_quick_guide.htm

Scroll down to the api section "Connect to the ZooKeeper Ensemble"

Infelicity

I tried the suggestion given by you. However it temporarily solved the problem and I again started facing the problem when load increased further. – Collapse