Difference between Zookeeper and a managed replicated database service

Question

I just came across Zookeeper and am wondering as to what's the difference between Zookeeper and an available, consistent, durable, distributed, replicated database service like AWS DynamoDB or even AWS S3(storage service) for that matter. The key features like configuration management, distributed synchronization etc can very well be achieved with a database offering like AWS DynamoDB. I understand that there would be architectural differences between Zookeeper and products like DynamoDB. But, from a feature perspective, are there any major differences between the two ? Is there any reason to use Zookeeper over the other.

Explanatory

see this post – Tritium

Answer 1

First let me tell you some basics about zookeeper which you may already know:

• Zookeeper is not a database
• Zookeeper is a coordination service
• Zookeeper is highly available and capable of managing more than 4000 nodes in a cluster.
• Zookeeper stores all its information in znodes, and every Znode can be of 1 mb max.
• Zookeeper provides 3 types of znodes: ephemeral, sequential and persistence

Now, to answer your query:

Zookeeper is used for providing exclusive locks to the services where there is a master-slave architecture and you want only one service to be active and perform all the reads/writes.

Zookeeper can be used for sessions also. Like an ephemeral node will be generated per user for session and when the user logs out, the node will automatically be deleted from the zookeeper memory.

Zookeeper is reliable and fault-tolerant and performs in-memory operations which makes it even faster.

So, there are the main reason why zookeeper is considered above any other services providing coordination.

Unmoor

Answer 2

Zookeeper in a nutshell if a distributed kernel, it provides low primitives using which you can build complex DISTRIBUTED SYSTEMS further.

1) Zookeeper provides ordered messages, which is very required for distributed locks(distributes systems in general). Dynamo db does not provide ordered message per client guarantee.

2) Sequential znode provide atomic way to add elements in a ordered way with a common prefix string. Combined with Ephemeral nodes and ordered notification they let you create notification.

lets say you want to lock a customerABCD to perform a work, every machine can write Create('/customerABCD/lock-', Sequential) if there are 2 nodes performing above Create then znodes formed will be /customerABCD/lock-1 & /customerABCD/lock-2.

To decide who is leader you can simple query Get('/customerABCD') key and then decide leader with least key value. Now lets say Node which created lock-1 dies, then lock-2 will get notification message from zookeeper and then it can claim ownership of customerABCD. More examples of such distributed tasks are in https://learning.oreilly.com/library/view/zookeeper/9781449361297/ch02.html

In Dynamo machine which created /customerABCD/lock-2 znode will have to poll to know if lock exists or not. This is slow way to acquire lock as it requires timeout based polling, this is inefficient as compute is required to perform poll as well, and adds polling load to system as well.

3) when znodes are added/removed then zxid version gets incremented. This forms the basis of versioning which can be used by distributed systems to achieve lock with fencing as explained in "Making the lock safe with fencing" in link https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html

Again Dynamo does not seems to have similar auto-increment parent sequence number facility.

Parboil

What do you mean by Dynamoc does not provide ordered messages per client? – Omora

Time_T1 -> Write x = 5 Time_T2 -> write y = 9 _____________________________________ Events received by clients can be Y = 9 and then X = 5 ordering of events received by clients can be different while using DynamoDB – Parboil