The idea: one key in Redis decides who may work
Several workers, on different machines, must not update the same thing at the same time. A distributed lock lets one of them in. With Redis the lock is a single key. To take it, a client writes the key only if nobody else has, with an expiry:
SET lock:order-42 B:91ab NX PX 10000 → OK (you hold it)
→ (nil) (someone else does, try later)
On the canvas, clients A, B and C each want to add 1 to a counter in a database. Each one takes the lock, reads the counter, writes counter + 1, and releases. If the lock works, three jobs leave the counter at 3. The timeline at the bottom shows, second by second, which client believes it holds the lock, who really holds the key in Redis, and a red ✗ in the unsafe row whenever a running client acts on a lock it no longer has.
NX, PX and the random value
| part | why it is there |
|---|---|
NX | set only if the key does not exist: the check and the write are one atomic step, so two clients cannot both win |
PX 10000 | Redis deletes the key after 10 s. If the holder crashes, the lock frees itself instead of blocking everyone forever |
random value (B:91ab) | proves who set the key, so a release can check that the lock is still yours |
The expiry is what makes the lock safe against crashes, and also what makes it unsafe against slowness: the lock ends after 10 s whether the holder is done or not.
Releasing: check-and-delete, not DEL
A plain DEL lock:order-42 deletes the key no matter who holds it now. If your lock already expired and someone else took it, your DEL removes their lock, and a third client can walk in while they still work (Demo 4: slow job + plain DEL: A deletes B's lock, C gets in, then B deletes C's). The fix is a small Lua script, which Redis runs atomically:
if redis.call("GET", KEYS[1]) == ARGV[1] then
return redis.call("DEL", KEYS[1])
else
return 0
end
Doing GET and then DEL as two separate commands is not enough: the key can expire and change owner between them.
The pause problem
A client cannot check its lock before every single instruction. Between "I hold the lock" and the write to the database there is always a gap, and a process can stop in that gap for longer than the TTL: a stop-the-world garbage collection, a VM migration, swapping, a slow network. When it wakes up, nothing tells it that time passed. In Demo 2: GC pause, no fencing A is frozen from t = 1 to t = 15. Its key expires at t = 10, B takes the lock at t = 11 and writes counter = 1. A wakes up and writes counter = 1 too, based on the 0 it read before the pause. Three jobs, counter = 2: an update is lost.
A slow job does the same without any pause (Demo 5): A's work takes 12 s, the TTL is 10 s. A check-and-delete release keeps the locks themselves tidy, but A's write still lands while B holds the lock.
Fencing tokens
The fix moves the check to the place that is damaged: the storage. Every time the lock is granted, the holder also gets a fencing token, a number that only grows (here INCR lock:fence: 33, 34, 35). The client sends the token with every request. The storage remembers the highest token it has seen and refuses anything smaller. In Demo 3 B's requests carry 34, so when A wakes up and writes with 33, the storage answers rejected: 33 < 34. A gets an error instead of silently corrupting data. In Demo 6 the storage checks reads too: B's read with 34 at t = 11 already fences A off before A's late write.
Where the token comes from:
| lock service | fencing token |
|---|---|
| Redis | a separate INCR counter, as on this page (a script can do SET NX and INCR together) |
| etcd | the key's revision (create_revision), which grows with every write to the cluster |
| ZooKeeper | the zxid or the sequence number of the lock's ephemeral node |
| a database row lock | no token needed if the work happens in the same transaction that holds the lock |
The storage must support the check: a conditional update (UPDATE … WHERE last_token < 34), a compare-and-set, or a version column. If it cannot check, the lock can only make overlaps rare, not impossible.
One Redis node, replicas, Redlock
With a single Redis node, the lock is gone if that node dies. With a replica, replication is asynchronous: the primary can say OK and fail before the key reaches the replica, the replica is promoted (see Redis Sentinel), and a second client gets the same lock. Redlock asks five independent Redis nodes and counts the lock as taken only with a majority, inside a time budget. It survives a node failure, but it still relies on timing: a pause or clock jump can still make two clients believe they hold it. It still needs fencing for correctness.
| you need the lock for | what to use |
|---|---|
| efficiency: avoid doing the same work twice; a rare overlap only costs time | one Redis node with SET NX PX and check-and-delete is fine |
| correctness: an overlap corrupts data or sends money twice | a consensus store (etcd, ZooKeeper) and fencing tokens checked by the storage, or the database's own transactions |
Long jobs can renew the lease before it ends (a watchdog that extends the TTL, an etcd lease keep-alive), which makes expiry rarer. It does not remove the pause problem: the renewal can be paused too.
See also Mutex for a lock inside one machine, where the kernel knows exactly who holds it, and Primary–Replica Replication for fencing an old primary.
The etcd version of a lock (a lease plus a transaction on create_revision) is animated on etcd: the Store Behind Kubernetes.
What the page leaves out
Lease renewal and watchdogs, Redlock's majority vote and clock drift, waiting with pub/sub instead of polling, fairness (who gets the lock next), reentrant locks, the network delay of each request (every request here takes no time), and a crashed client: its lock simply expires after the TTL.