How do one read the Zookeeper transaction log?

Question

Are there any existing tools that help to read the Zookeeper transaction log? By default, it is in binary format and I would like to read it in human readable form.

Whish

Answer 1

I don't know if you have solved this question.

cd /your/zookeeper/directory/

If you want to read snapshots, use:

java -cp zookeeper-3.4.6.jar:lib/log4j-1.2.16.jar:lib/slf4j-log4j12-1.6.1.jar:lib/slf4j-api-1.6.1.jar org.apache.zookeeper.server.SnapshotFormatter version-2/snapshot.xxx

If you want to read logs, use:

java -cp zookeeper-3.4.6.jar:lib/log4j-1.2.16.jar:lib/slf4j-log4j12-1.6.1.jar:lib/slf4j-api-1.6.1.jar org.apache.zookeeper.server.LogFormatter version-2/log.xxx

Barimah

4 years later and your post is still helpful! Thanks a bunch Oliver and Tony. I was attempting to run this command on the server log inside of the logs folder (not realizing I was meant to run it on the transaction logs...) – Colander

with zookeeper 3.5.6 this worked for me (updated jar names and had to add jute): java -cp lib/zookeeper-3.5.6.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.25.jar:lib/slf4j-api-1.7.25.jar:lib/zookeeper-jute-3.5.6.jar – Gassman

The LogFormatter tool is removed in 3.7 – Tion

Answer 2

You can use something like this

java -cp $ZOOKEEPER_CLASSPATH org.apache.zookeeper.server.LogFormatter [zookeeper log file path]

Scattering

I can't make it work. Keeps returning Error: Could not find or load main class log.5200768f38 – Moist

@Eye - You need a valid value for ZOOKEEPER_CLASSPATH set before you can execute Vikas's command. I personally prefer to simply reuse HBase's classpath setup: hbase org.apache.zookeeper.server.LogFormatter <filename> which automatically loads the class. – Catheryncatheter

Answer 3

Building on the previous two answers, using Zookeeper 3.5.6: from the /path/to/zookeeper/lib dir which contains all the ZK and supporting jars run:

java -cp * org.apache.zookeeper.server.LogFormatter /path/to/zookeeper/transaction/logs/version-2/log.xxx

Gassman

Answer 4

Since zookeeper version 3.6 there are tools to read transaction log and snapshots in zookeeper distribution:

For transaction log:

bin/zkTxnLogToolkit.sh --dump /datalog/version-2/log.f3aa6 

For snapshots:

./zkSnapShotToolkit.sh -d /data/zkdata/version-2/snapshot.fa01000186d

See the details in official docs

Tion

Answer 5

You can enable ZooKeeper audit logs for ZooKeeper 3.6.o and upwards. To enable audit logs configure audit.enable=true in conf/zoo.cfg.

One thing to keep in mind is that logs from different servers of the same ensemble should be aggregated because each one of them includes operations that where executed only form clients connected to this particular server.

Full information here: https://zookeeper.apache.org/doc/r3.6.1/zookeeperAuditLogs.html

Bacillary