Replication

Storing copies of the same data on multiple nodes so the system is fault-tolerant, can scale reads, and can serve data from locations closer to users.

The leader accepts writes and is the source of truth; followers receive a stream of changes from the leader and serve reads (and can be promoted to leader on failure).

• Availability/durability: survive a node failure without data loss.
• Read scalability: distribute reads across replicas to handle more traffic.

Replication keeps live, continuously-updated copies for availability and scaling; it also propagates mistakes (a bad DELETE hits all replicas). A backup is a point-in-time snapshot kept for recovery from data corruption or accidental deletion. You need both.

The time delay between a write being applied on the leader and appearing on a follower. During this window, reads from the follower return stale data.

Only the leader accepts writes. Followers are read-only copies; write requests routed to a follower are rejected or forwarded to the leader.

The process of promoting a follower to be the new leader when the current leader fails, so the system keeps accepting writes. It can be manual or automatic.

Synchronous: the leader waits for follower acknowledgment before confirming the write — guarantees the follower has the data (no loss on failover) but adds latency and stalls if a follower is slow/down. Asynchronous: the leader confirms immediately and replicates in the background — low latency and resilient to slow followers, but a leader crash loses writes not yet replicated. Many systems use semi-synchronous (wait for at least one follower).

A user may write then read a stale follower ("I updated my profile but it shows the old value"). Handle with: read-your-writes (route a user's reads to the leader for a short window after their write, or to a replica known to be caught up), monotonic reads (pin a user to one replica so time doesn't appear to go backward), and monitoring lag to route reads only to sufficiently fresh replicas.

Multiple nodes accept writes and replicate to each other. Used for multi-datacenter deployments (each region has a local leader for low-latency local writes), offline-capable clients, and collaborative editing. The cost is write conflicts when the same data is edited in two leaders concurrently, which must be detected and resolved.

A model (Dynamo-style, e.g. Cassandra) where any replica accepts writes and there is no single leader. Clients (or a coordinator) send reads and writes to multiple replicas; quorums and mechanisms like read repair and anti-entropy keep replicas converging. It offers high write availability and no failover step, at the cost of tunable/eventual consistency and conflict handling.

Adding followers lets you spread read queries across many nodes, scaling read throughput. Limits: it only scales reads (all writes still go through one leader), replication lag means reads can be stale, and every replica must apply the full write load, so a write-heavy workload eventually saturates each replica regardless of count. For write scaling you need sharding.

Statement-based ships the SQL statements to replicas to re-execute — compact but unsafe for nondeterministic functions (NOW(), RAND()), auto-increment, and triggers, which can diverge. Row-based / log-based ships the actual changed rows (from the write-ahead log/binlog) — deterministic and robust, at the cost of larger replication streams. Most modern systems default to row/log-based for correctness.

The WAL is an append-only log of every change written before it's applied to the data files, ensuring durability and crash recovery. Replication commonly works by streaming this log to followers, which replay it to stay in sync (log shipping / streaming replication). It also enables point-in-time recovery and change-data-capture.

With N replicas, if every write is acknowledged by W replicas and every read queries R replicas, then setting W + R > N guarantees at least one replica in any read overlaps the latest write, so reads can find the newest value (using version metadata to pick it). Common config: N=3, W=2, R=2. Tuning W/R trades consistency vs availability/latency: higher W favors durable writes, higher R favors fresh reads; W=1,R=1 is fast but weakly consistent. Note quorums still don't handle concurrent-write conflicts by themselves.

Split-brain occurs when a network partition (or a wrongly-triggered failover) leaves two nodes both acting as leader, each accepting conflicting writes that later can't be cleanly merged. Prevention: require a majority quorum to elect/retain a leader (a minority partition can't have a leader), use consensus (Raft/Paxos) for leader election, employ fencing tokens (monotonic epoch numbers) so storage rejects writes from a deposed leader, and use an odd number of nodes / witness to break ties.

Options: Last-write-wins (LWW) by timestamp — simple but silently drops concurrent writes and is sensitive to clock skew. Version vectors / vector clocks — detect concurrency and surface conflicting versions (siblings) for the application to merge. CRDTs — data types (counters, sets, maps) mathematically designed to merge deterministically without conflicts. Application-defined merge — custom logic (e.g. union shopping carts). The right choice depends on whether losing a concurrent write is acceptable.

Mechanisms to converge replicas in leaderless systems. Read repair: when a read detects that some replicas returned stale values, the coordinator writes the latest value back to the stale replicas in the background — fixing frequently-read data. Anti-entropy: a background process (often using Merkle trees to efficiently find differences) continuously compares and reconciles replicas, catching rarely-read data that read repair would miss. Together they ensure eventual consistency.

Writes acknowledged by the old leader but not yet replicated are lost when a follower missing them is promoted. Minimize with semi-synchronous replication (wait for at least one follower to persist each write), promoting the most up-to-date follower (highest log position), and using consensus-based replication so a committed write is guaranteed on a majority before ack. There's an inherent trade-off: stronger durability (sync) costs write latency and availability. Some systems keep the old leader's un-replicated writes to reconcile after it rejoins.

Raft elects a single leader via majority vote (terms/epochs prevent two leaders). The leader appends each command to its log and replicates to followers; an entry is committed once a majority have persisted it, and only then applied to the state machine and acknowledged. Because commits require a majority and log-matching rules enforce identical, ordered logs, all nodes apply the same commands in the same order — a linearizable replicated state machine. A minority partition can't elect a leader or commit, preventing split-brain at the cost of unavailability without a majority.

Track per-user write position (e.g. the leader's log sequence number or a logical timestamp returned on write). On subsequent reads, either route to the leader/primary for that user's shard, or select a replica whose applied position is ≥ the user's last-write position (wait or retry if none is caught up). Store the token client-side (cookie) or in a session store. Across shards, track a position per shard the user has written. This provides causal/read-your-writes consistency without forcing full global strong consistency.

Nodes form a chain: writes enter at the head, propagate node-by-node to the tail, and are acknowledged from the tail; reads are served by the tail (which has all acknowledged writes), giving strong consistency. Pros: strong consistency with good read throughput and simple recovery rules. Cons: write latency grows with chain length, the tail is a read hotspot, and reconfiguration on node failure needs a separate coordinator (e.g. a master/ZooKeeper). Variants like CRAQ let all nodes serve consistent reads to spread read load.

Cross-region links are high-latency and less reliable, so synchronous replication across continents kills write latency. Common patterns: async cross-region replication with a primary region for writes and read replicas elsewhere (accepting some lag and potential loss on region failure); multi-leader per-region with conflict resolution for local low-latency writes; or consensus systems that place replica majorities carefully (e.g. Spanner uses Paxos groups with TrueTime). Also consider data residency/compliance, failover RPO/RTO targets, and routing users to their nearest region while keeping a clear source of truth per data partition.

A follower copy of the database that serves read queries only, offloading reads from the primary. Applications route reads to replicas and writes to the primary to scale read-heavy workloads.

No — read replicas are read-only. Writes must go to the leader/primary, which replicates them to followers. Attempting to write to a replica is rejected (or, in some setups, transparently forwarded to the primary).

A middle ground where the leader waits for acknowledgment from at least one (but not all) follower before confirming a write. It guarantees the write survives on more than one node (no single-node data loss) without the latency and fragility of waiting for every replica. If the chosen follower is slow/down, the system can fall back to async temporarily.

A topology where followers replicate from other followers instead of all pulling directly from the leader, forming a tree. This offloads replication fan-out work from the leader (useful with many replicas or cross-region setups) at the cost of increased replication latency for downstream replicas and more complex failure handling.

Physical (binary) replication ships low-level storage changes (WAL/blocks) so replicas are byte-identical copies — efficient but tightly coupled (same version/platform, whole cluster). Logical replication ships change events (row inserts/updates/deletes) that replicas re-apply, allowing selective tables, cross-version replication, and transforming data — enabling zero-downtime upgrades and feeding downstream systems, at higher overhead and some feature limits.

Measure lag both as time (seconds behind primary) and as position (bytes/LSN between primary's write position and each replica's applied position). Expose per-replica lag metrics, alert when lag exceeds thresholds that break read-your-writes routing or approach failover RPO limits, and watch for lag trends (steadily growing means the replica can't keep up — I/O bound, long-running queries blocking apply, or write spikes). Also monitor replica apply errors and connection health. Route reads only to replicas within an acceptable freshness bound, and take a lagging replica out of the read pool automatically.

Use logical replication to bridge versions. Steps: stand up a new-version replica and start logically replicating from the old primary so it stays current; let it catch up and verify data parity (row counts/checksums, and shadow reads comparing results); then do a brief, controlled cutover — stop writes momentarily (or use a proxy to pause), ensure the new node is fully caught up, promote it to primary, and repoint the application/proxy to it; keep the old primary as a fallback (reverse replication if possible) for quick rollback. Because logical replication tolerates version differences, the old and new versions run simultaneously during the migration, keeping downtime to the seconds-long cutover. Tools like pglogical / native logical replication or managed blue-green deployments implement this.