Zookeeper

tags : Infrastructure,Distributed Systems

Links §

• ClickHouse Keeper: A ZooKeeper alternative written in C++

What? §

• Fundamentally a distributed in-memory hierarchical key value store
• Usecases: Leader election, Distributed locks, implementing Paxos etc.
• Eg. Zookeeper is used by Kafka brokers to determine which broker is the leader of a given partition and topic and perform leader elections.

Data Model §

• Data model looks like a filesystem(znodes).
• More like a metadata/lock server that other applications can use for various usecases

Architecture §

• Leader-follower architecture
• All nodes(Ensemble) participate in decision
• A minimum of 3 nodes are recommended for production.
• Has something called watch: Async, one time trigger that gives listners the ability to be notified when something changes in a znode, can be triggered on the value/status change of its children.

Guarantees §

• Eventually consistent system, there is possibility of stale reads.

Deployment §

• Can use k8s replicas to create zookeeper quorum/ensemble
• 1 k8s node can probably run multiple zk nodes even if the official tutorial uses pod affinity
• There’s no 1-1 relation but then again you’ll need to play with port numbers.

Snippets §

# ports
# :2181 client port, other applications connect to this port to use zk.
# :2888 follower port
# :3888 election port
# :8080 AdminServer port
 
# local docker run
$ docker run --name zuuu --rm -p 8080:8080 -p 2181:2181 -p 2888:2888 -p 3888:3888 zookeeper

Components §

z-node §

• A byte array, it has timestamps, value, ACLs, optional children among other things.
• Session: Connection between a client and node in the ensable, the client can talk to any of the followers or the leader, when a session happens an epheremal node can be created in the context of that session.