Understanding Distributed Systems 1.1

Coordination is expensive as it reduces the availability and performance of distributed applications (PACELC theorem). I have extended chapter 10 of Understanding Distributed Systems with a discussion of how to minimize coordination using one of the following patterns:

• Keep coordination off the critical path
• Use protocols that guarantee some form of consistency without coordination
• Proceed without coordination and apologize when an inconsistency is detected

Section 10.4 describes chain replication, a widely used replication protocol that guarantees strong consistency. It has better performance than Raft because it keeps consensus off the critical path by delegating fault-tolerance to a dedicated “control plane.”

Section 10.5 is all about solving the CAP theorem using weaker consistency models. It discusses how to build eventually consistent replicated data types that converge to the same state (CRDTs) and shows how these ideas are used in practice to build distributed databases (Dynamo-like data stores). The CALM theorem is also introduced to define which applications are safe under eventual consistency.

Finally, the section explores causal consistency, a consistency model that is stronger than eventual consistency but weaker than linearizability. It’s particularly attractive for two reasons:

• For many applications, causal consistency is “consistent enough” and easier to work with for developers than eventual consistency
• Causal consistency is provably the strongest consistency model that solves the CAP theorem. In other words, the model is as consistent as it gets while still being available under network partitions