Foundations of Scale
Caching and the CDN
Your database answers a question in 50 milliseconds. Memory answers the same question in 50 microseconds — a thousand times faster. Every slow system you have ever used was mostly a system that kept asking the database questions it had already answered.
Reads dominate almost every system, and most of them are repeats: the same profile, the same product page, the same leaderboard. Sending every repeat to the database wastes its most precious resource — and at scale, waste becomes an outage.
- On read: check the cache first. If the data is there (a hit), return it — the database never wakes up.
- On a miss: read from the database, return the result, and write a copy into the cache. The next reader gets the fast path.
- On write: update the database first, then invalidate the cached copy. The next read misses once, reloads the fresh value, and re-populates the cache.
Cache-aside makes your application own population: the cache is a dumb store, and the app decides what goes in and when it dies. Read-through flips that — the cache itself loads from the database on a miss, so your code just asks the cache; simpler call sites, but now the cache must understand your data source. Write-through sends every write through the cache to the database synchronously: the cache is always fresh, but every write pays full database latency. Write-behind goes further — the cache acknowledges the write instantly and flushes to the database in batches later: writes get blazing fast, and if the cache node dies before the flush, that data is gone. Speed, freshness, durability — you never get all three for free.
Every cached value has a TTL — a time-to-live after which it expires. Short TTL means fresher data and more database load; long TTL means the opposite. When the cache fills up, an eviction policy like LRU throws out the least recently used entries. And beware the hot key: if a celebrity's profile expires at a traffic peak, a thundering herd of requests slams the database at once — stagger expirations or lock so one request reloads while the rest wait.
Sharding your cache spreads load across nodes — until one key goes supernova. A celebrity's post or a flash-sale item can pull millions of requests per second, and they all hash to the same cache node; no amount of sharding helps a single key. The fix is key-local caching: keep a tiny in-process L1 cache inside each application server for that handful of ultra-hot keys, so reads never leave the process. The cost is staleness squared — hundreds of servers each hold their own copy, with no invalidation reaching them, so every copy drifts until its (very short) TTL expires. Use it only for keys where a few seconds of disagreement is acceptable, like view counts — never for a balance.
Static assets — images, videos, JavaScript bundles — should not travel from your one data center to every user on Earth. A CDN copies them to edge servers near your users and serves them from there, honoring the TTL headers you set. When you must replace an asset before its TTL ends, you invalidate it at the CDN explicitly. The physics is the same as a cache; the geography is new.
A push zone works like a warehouse you stock yourself: you upload files to the CDN's storage, and every edge serves your copy until you replace it. You control exactly what users see and your origin carries zero traffic — ideal for large, rarely-changing files like videos and installers, at the price of doing the publishing work yourself. A pull zone works like a shop that orders on demand: the CDN fetches from your origin on the first miss at each edge, caches it, and serves repeats. Setup is trivial — point the CDN at your origin and walk away — but the first user in every region pays the slow round trip, and your origin must stay alive and fast enough for cache fills. Push for control, pull for convenience.
A cache is not a small database — it is a bet that some data is read far more often than it changes. Place that bet only where the odds are good, and always know what you lose when the bet goes stale.
Never treat the cache as the source of truth. Caches lose data, expire entries, and can be wiped by a restart — the database is the truth, the cache is a shortcut. Any write path that updates the cache but not the database is a data-loss bug waiting for its moment.
In cache-aside, why do writes invalidate the cache instead of updating it — and what goes wrong if you update the cache but forget the database? (Invalidating avoids races between concurrent writes and keeps one source of truth; forgetting the database means a cache flush silently erases user data.)
Memory beats disk by a thousand, so answer repeats from memory: check cache → miss → database → populate; on write, update the database and invalidate. Choose staleness deliberately with TTLs, protect hot keys from stampedes, and let a CDN do the same trick for static files near your users.
Find the three most repeated reads in a system you own. For each, write down: how often it changes, what a stale answer would cost, and what TTL you could live with. That is your caching shortlist.
Foundations of Scale