// Foundations of Scale
Data at scale: replicate, shard, distribute
System Design track · 20 min
- Tell whether a struggling database needs replication, sharding, or both.
- Pick a shard key and explain the trade-off between range and hash sharding.
- Hide replication lag from users with read-your-writes tricks.
Remember the bottleneck from the first lesson? Your database just became it, again. But 'the database is struggling' can mean three different things.
Maybe reads are drowning it. Maybe the data no longer fits on one machine. Or maybe it is one hardware failure away from taking the company down. Three different illnesses, and each has its own cure.
There are two tools, and they do different jobs.
Replication keeps full copies of the same data on several machines. It gives you more read capacity and lets you survive a machine dying. Sharding splits different parts of the data across several machines. It gives you more write capacity and more room.
Think of a government office. Replication is printing the same citizen register for every branch, so any branch can answer questions. Sharding is splitting citizens by surname letter, A to F at counter one, G to M at counter two, so no single counter holds everyone. Large systems usually end up doing both: each shard has its own copies.
In leader-follower replication, one leader takes every write and streams the changes to followers, which answer reads. Need more reads? Add a follower.
The key choice is when a write counts as done. Synchronous replication waits until the followers have it: no write is ever lost, but every write is slower. Asynchronous replication confirms right away and copies a moment later: fast, but if the leader crashes, the newest writes can go with it.
When the leader dies, a follower is promoted to take its place (failover). That is exactly the moment any delay in copying becomes visible to users.
With asynchronous copying, a write reaches the followers a few milliseconds, sometimes a few seconds, after the leader. So a user edits their profile, refreshes, gets sent to a follower that is behind, and sees the old value. Their own change has vanished. This is the read-your-writes problem.
Its quieter cousin: two refreshes hit two followers that are behind by different amounts, and the data seems to go back in time.
The fixes are cheap and boring. Send a user's own reads to the leader for a short while after they write. Or have the client remember when its last write happened and only read from a follower that has caught up. Or keep each user on one follower, so their reads never go backwards. None of this removes the delay; it hides it from the people who would notice.
When one database cannot hold the data or keep up with the writes, split it. Users with IDs 1 to 1 million go to server A, 1 million to 2 million to server B, and so on. The field you split by is the shard key, and choosing it is the whole game. A bad key piles load onto one server instead of spreading it.
The classic failure is the celebrity problem: one user's data is so busy that their shard overheats while the others sit idle. And changing the key later means moving rows between live servers, one of the most painful jobs in engineering. Choose the key as if you will live with it for years. You will.
Range sharding gives each shard a continuous slice of keys: names A to F here, G to M there. The win is ordered reads: 'all users from A to C' stays inside one shard and stays fast. The cost is hot spots. Shard by an increasing ID or by date, and every new write lands on the last shard while the rest do nothing.
Hash sharding runs the key through a hash function, which turns it into a number that looks random, and uses that number to pick a shard. Writes spread almost perfectly evenly. The cost: order is gone, and 'all users from A to M' must ask every shard and merge the answers.
Pick range when ordered scans matter most and you can handle rebalancing. Pick hash when an even spread matters more than order.
Here is the catch with hash sharding. The obvious rule, hash(key) mod N where N is the number of servers, falls apart the moment N changes: add one server and almost every key now points somewhere else.
The standard trick is consistent hashing. Put both servers and keys on the same circle using the same hash function. Each key belongs to the first server you meet going clockwise. Add or remove one server and only about 1/N of the keys move; the rest stay put. This idea quietly powers caches, sharded databases and load balancers everywhere.
Try it: add and remove servers in both modes, the ring and plain mod N, and compare the share of keys that move.
The naive reshard, changing N in hash(key) mod N, moves almost every row and takes the system down with it. The standard answer is to create many logical shards up front, say 1,024, and place them on far fewer real machines, say 8 (128 each). Growing to 16 machines means handing 64 logical shards from each old machine to a new one. That is a change to a routing table, not surgery on your data model.
During the move you write to both places: every new write goes to the old and the new location, while a background job copies the existing rows. Reads switch over only after the copy is checked.
The rule that saves you years: start with many more logical shards than machines, so growth is always 'give some shards to the new machine', never 'redesign the split while under load'.
Start with SQL. Transactions (a group of changes that succeed or fail together), joins, constraints and forty years of hard-won reliability cover most apps comfortably.
Reach for NoSQL when the scale or the access pattern really demands it: huge write volumes, a schema that keeps changing, simple key-value lookups at massive volume, or data that is naturally a document or a graph. SQL is the default. NoSQL is the exception you justify with numbers, not fashion.
Your app is 95% reads and its one database is dying. Replication or sharding? And which one helps when the data simply no longer fits on one disk?
Answer in your head first, then check.
Copy the data to survive and to read more. Split the data to write more and to hold more. Default to SQL, justify NoSQL with numbers, and use consistent hashing whenever keys must spread across servers that come and go.
Take the largest table in a system you know. Write down: what you would shard it by, which rows would be the 'celebrity', and which of your queries break the moment the data lives on two machines. If the answers scare you, you now know why sharding is the last resort.
- Tell whether a struggling database needs replication, sharding, or both.
- Pick a shard key and explain the trade-off between range and hash sharding.
- Hide replication lag from users with read-your-writes tricks.
Foundations of Scale