How to persist Zookeeper data in a docker setup via a volume?
Asked Answered
C

1

6

So, I am setting up Zookeeper in my local environment on docker using the official docker image. Now, what I want with this Zookeeper setup is to be able to persist the nodes and their values by mounting the Zookeeper data directory as a volume inside my container.

This is straightforward, but what's confusing me is that Zookeeper's data directory inside the container is at /data while the datalog directory is at /datalog. Ideally, I would expect Zookeeper to persist the nodes and values inside the data folder as well in addition to the datalog directory. However, I find that while the datalog directory has the data in log.* files, the data directory does not have any data related to the test nodes that I created.

bash-4.4# pwd
/datalog/version-2
bash-4.4# ls
log.1  log.3  log.5  log.8


bash-4.4# pwd
/data/version-2
bash-4.4# ls
bash-4.4# 

My goal is achieved with this Zookeeper service in my docker-compose.yml -

volumes:
  - zkdata:/datalog

but this seems counter-intuitive that I am mounting the transaction logs directory instead of the actual data directory. How and where does Zookeeper actually persist the node data and how can I correctly mount it as a volume in my docker container?

I can actually set the ZOO_DATA_LOG_DIR environmental variable to /data and mount that as a docker volume, but my question is really about why the Zookeeper data is empty while the datalog directory isn't.

Cattery answered 18/9, 2018 at 4:21 Comment(1)
A bit late but maybe useful for future readers. I was studying this tutorial: Setting up an Apache Zookeeper cluster in Docker. It has information about setting up zookeeper with volumes via docker commands. Maybe it helps.Public
K
2

Probably you should see the dataDir path in zookeeper config and mount that path in the volumes. In my case zookeeper config is present at /config/zoo.cfg

dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
server.1=localhost:2888:3888;2181

dataDir is the place where zookeeper stores the in-memory database snapshots which will be referred in case of failure.

Since the dataDir is /data so I mounted /data directory in volumes so the entry would be :

volumes:
  - zkdata:/data
Keil answered 5/12, 2019 at 10:0 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.