Zookeeper znode count

Question

Here is the output of zookeeper monitoring

    zk_version  3.4.6-1569965, built on 02/20/2014 09:09 GMT
    zk_avg_latency  0
    zk_max_latency  0
    zk_min_latency  0
    zk_packets_received 3
    zk_packets_sent 2
    zk_num_alive_connections    1
    zk_outstanding_requests 0
    zk_server_state follower
    zk_znode_count  16349
    zk_watch_count  0
    zk_ephemerals_count 6
    zk_approximate_data_size    19502850
    zk_open_file_descriptor_count   30
    zk_max_file_descriptor_count    4096

I would like to understand what zk_znode_count refers to and also I want to keep (zk_znode_count & zk_approximate_data_size) values minimal to avoid sync issues with followers.

Could someone throw some insight on these values

Burgoo

Answer 1

zk_znode_count is the total count of znodes stored in the ZooKeeper ensemble. Every time a client creates a new znode, this counter will increment. Every time a client deletes a new znode (either explicitly or by dropping its ephemeral znodes after disconnection), this counter will decrement.

zk_approximate_data_size is the approximate memory consumption for all znodes stored in the ZooKeeper ensemble. It's called an approximation, because it may fail to account for some overhead factors in the internal data structures. (The current implementation is a Java ConcurrentHashMap mapping String paths to the znode data and some metadata.) For large-scale usage, it's good to monitor zk_approximate_data_size and make sure it doesn't get too close to exhausting the allocated JVM heap size.

Barbel

How to restrict zk_approximate_data_size. Say I want to keep znodes that are only one week old or I want to have max 50 MB of zk_approximate_data_size. Can I get the znodes that are older than a week using zkCli – Burgoo

ZooKeeper tracks a creation time and modification time for each znode. Those are visible when you run commands like get in the CLI and accessible through the ZooKeeper Java client API methods. There is no specific command to query for only znodes of a certain age, so you'd have to scan them yourself and check the creation time and modification time. Apache JIRA ZOOKEEPER-2169 tracks implementation of TTL on znodes, but that isn't shipping until version 3.6.0, so that's more future-looking. There are no restrictions based on size. – Barbel