Heavy Lifting + Your Rehearsal
Design a file-sync service
You save a 1 GB video on your laptop, and thirty seconds later it is on your phone. Your editor touched 10 MB of it. Somewhere, a system decided to move 12 MB instead of 1 GB — and that decision is the entire design. Everything else is bookkeeping.
Do not sync files — sync blocks. Split every file into fixed-size chunks (say 4 MB) and hash each one. Now an unchanged block is never re-uploaded, because its hash already lives on the server. And an identical block appearing in a hundred files is stored exactly once, because the hash is its address. This single choice — chunk and hash — defines the bandwidth bill, the storage bill, and the sync speed of the whole system.
A 1 GB file at 4 MB per block is 250 blocks — 250 hashes, a few KB of metadata. Edit 10 MB in the middle: at most 3 blocks change, so the client uploads ~12 MB plus the new hash list, not 1 GB. Storage side: if 10,000 users all keep the same OS installer, dedup stores it once, not 10,000 times. Content-addressed storage turns duplication into a counter, not a copy.
- Metadata service: file name → ordered list of block hashes, versions, permissions, sharing. Small, hot, and it must be right — put it on a relational, strongly-consistent store.
- Block storage: the actual bytes, addressed by hash. Huge, cold, and immutable — put it in cheap object storage, where a terabyte costs a fraction of what the database costs.
- Why split them? Metadata is kilobytes per file but needs transactions and consistency — a wrong block list corrupts a file. Blobs are gigabytes but need nothing but durability — a hash already guarantees integrity. Matching each to its cheapest correct home is the whole cost model.
- Push: a client-side watcher notices local changes, re-chunks the changed files, compares hashes against the last known list, and uploads only the blocks the server lacks.
- Pull: the client asks the metadata service what changed since its last sync, downloads the missing blocks, and reassembles files locally.
- Notify: a user's other devices hear about changes via long polling or push notifications — the message carries no data, just 'something changed, come pull.' Cheap, and it scales because the heavy lifting stays in the pull.
Last-write-wins at the file level: the later edit becomes the current version. It sounds brutal, but nothing is silently lost — version history keeps the loser, so the user can resurrect it. Block-level merging of two concurrent edits is a research project; file-level last-write-wins plus history is a product. Choose the one you can ship and defend.
Laptops close mid-edit; phones lose signal in elevators. Edits made offline queue locally — the client keeps chunking and hashing as usual, and when the connection returns, it replays the queue: upload missing blocks, commit the new metadata, resolve any conflicts against versions that landed meanwhile. The user never sees a difference between 'saved' and 'synced' until they look for it.
Sync is a metadata problem riding on a blob problem. The blobs are easy once they are content-addressed: dedup, integrity, and delta upload all fall out of the hash. Everything humans argue about — versions, conflicts, sharing, offline — lives in a few kilobytes of metadata per file. Get the block hashes right and everything else is bookkeeping.
A 1 GB video gets 10 MB edited — what actually crosses the wire, and why? (With 4 MB blocks, the edit touches at most 3 blocks; the client re-hashes the file, finds only those blocks missing on the server, and uploads ~12 MB plus a tiny metadata commit — a few KB. The other 247 blocks never move, because their hashes already match.)
The interview answer in one breath: chunk files, hash blocks, dedup storage; metadata in a consistent relational store, bytes in cheap object storage; clients push deltas and pull on notification; last-write-wins with version history; offline edits queue and replay. Every piece earns its place from the first decision.
Take any large file you edited recently and estimate the sync cost both ways: whole-file upload versus 4 MB blocks. Write a tiny script that chunks the file, hashes each block (SHA-256), and counts how many hashes change between two versions — then compare the bytes that would move. You will never design file sync the naive way again.
Heavy Lifting + Your Rehearsal