I'd like to programatically setup zookeeper cluster. My goal is to use machines with CoreOS and dynamically deploy three nodes in form of docker containers and setup them to one zoo cluster.
Except common setup in manual (/zookeeperReconfig.html) which shows how to add another nodes to existing three nodes cluster, I found a conversation which say how to do that from the beginning when I have no running nodes in existing cluster. Unfortunately, this set of steps does not work for me. I'm talking about http://mail-archives.apache.org/mod_mbox/zookeeper-user/201408.mbox/%3CCAPSuxQipZFH2582bEMid2tCVBFk%3DC31hwCYjbFgSwChBF%2BZQVQ%40mail.gmail.com%3E
Here is a list of steps I did:
- Run first node with standaloneEnabled=false and the only entry in zoo.cfg.dynamic.
server.1=localhost:2381:2281:participant;0.0.0.0:2181
- Run second node with following dynamic cfg:
server.1=localhost:2381:2281:participant;0.0.0.0:2181 server.2=localhost:2382:2282:observer;0.0.0.0:2182
Note that there is no difference in resulting behavior when I'd change "observer" to "participant" for second node.
- Now I have two running instances. I can use ./zkCli.sh to log into first node. When I try to add second node using following command:
reconfig -add server.2=localhost:2382:2282:participant;0.0.0.0:2182
... it fails with:
KeeperErrorCode = NewConfigNoQuorum for
However, after some research I found solution. But it's tricky and I don't think that it's the only correct solution.
What is working for me? I can do step #3 on first node again but now with "observer". This command causes that even first node knows about second node. When I type 'config' to console in zkCli, it seems that it's working. Next step is to log into second node using zkCli and than exec commands:
reconfig -remove 2 <- next step is not working w/o this
reconfig -add server.2=localhost:2382:2282:participant;0.0.0.0:2182
Well, now I have working cluster for two nodes. Finally, it's interesting that now I can add third node using regular scenario I've mentioned in first paragraph.
Do someone have some idea what I'm doing wrong?