// Heavy Systems + Your Rehearsal
Design a file-sync service
System Design track · 25 min
- Explain why file sync moves blocks, not files, and work out what an edit actually uploads.
- Split the design into a metadata store and a block store, and justify each one.
- Handle two devices editing at once, and edits made offline.
You save a 1 GB video on your laptop, and thirty seconds later it is on your phone. You only trimmed a few seconds, about 10 MB of changes. Somewhere, a system decided to move around 12 MB instead of 1 GB. That decision is the whole design. The rest is bookkeeping.
Do not sync files. Sync blocks. Cut every file into fixed-size pieces (say 4 MB) and compute a hash for each piece, a short fingerprint of its content.
Now a block that did not change is never uploaded again, because the server already has that hash. And a block that appears in a hundred files is stored once, because the hash is its address. This is called content-addressed storage: you find data by what it contains, not by where it sits.
This single choice, cut and hash, sets the bandwidth bill, the storage bill and the sync speed of the whole system.
A 1 GB file in 4 MB blocks is 1,000 ÷ 4 = 250 blocks. That is 250 hashes, a few KB of metadata.
Now edit 10 MB in the middle, overwriting it in place. A 10 MB stretch touches 3 blocks if it lines up well, and 4 if it starts near the end of a block. So the app uploads 12 to 16 MB plus the new hash list, not 1 GB. The other 246 or 247 blocks never move.
On the storage side: if 10,000 users keep the same installer file, the service stores it once, not 10,000 times. Duplication becomes a counter, not a copy.
- Metadata service: file name, the ordered list of block hashes, versions, permissions, sharing. Small, used constantly, and it must always be right. Put it in a relational database with strong consistency.
- Block storage: the actual bytes, found by hash. Huge, rarely read, and never changed. Put it in cheap object storage, where a terabyte costs a small fraction of the same terabyte in a database.
- Why split them? Metadata is kilobytes per file but needs transactions, because a wrong block list means a broken file. Blocks are gigabytes but need only durability, because the hash already proves the content is intact. Putting each in its cheapest correct home is the whole cost model.
- Push: a watcher on the device notices a local change, re-cuts the changed file, compares hashes with the last known list, and uploads only the blocks the server lacks.
- Pull: the device asks the metadata service what changed since its last sync, downloads the missing blocks, and rebuilds the files locally.
- Notify: the user's other devices learn about changes through long polling or push notifications, which you saw in the chat lesson. The message carries no data, just “something changed, come and pull”. It is cheap and it scales, because the heavy work stays in the pull.
Fixed 4 MB blocks work well when you overwrite bytes in place. But insert one byte at the start of a file, and every block boundary after it shifts by one byte. Every hash changes, and the whole file uploads again.
The fix is content-defined chunking: choose block boundaries from patterns in the content itself, not from fixed positions. An insertion then changes only the block around it. Tools like rsync use related ideas. In an interview, mentioning this one limit shows you understand the design, not just its happy path.
Use last-write-wins at the file level: the later edit becomes the current version. It sounds harsh, but nothing is silently lost, because version history keeps the other edit and the user can bring it back. Many services also save it as a “conflicted copy” next to the file.
Merging two edits block by block is a research project. File-level last-write-wins plus history is a product you can ship. Remember the key-value store lesson: LWW is safe only when losing a write is acceptable, and version history is what makes it acceptable here.
Laptops close in the middle of an edit, and phones lose signal in elevators. Edits made offline wait in a local queue. The app keeps cutting and hashing as usual. When the connection returns, it works through the queue: upload missing blocks, commit the new metadata, and resolve conflicts with any versions that arrived meanwhile.
The user should never notice the difference between “saved” and “synced” unless they look for it.
File sync is a metadata problem sitting on top of a storage problem. The storage part is easy once blocks are found by hash: dedup, integrity checks and delta uploads all come free with the hash. Everything people argue about, versions, conflicts, sharing, offline, lives in a few kilobytes of metadata per file.
A 1 GB video gets 10 MB edited in place. What actually crosses the network, and why?
Answer in your head first, then check.
The answer in one breath: cut files into blocks, hash them, store each block once; metadata in a consistent relational database, bytes in cheap object storage; devices push changed blocks and pull when notified; last-write-wins with version history; offline edits wait in a queue and replay. Every piece follows from the first decision.
Take a large file you edited recently and compare the sync cost both ways: whole-file upload versus 4 MB blocks. Write a small script that cuts the file into blocks, hashes each one with SHA-256, and counts how many hashes differ between two versions. Then compare the bytes each method would move. After that, you will never design file sync the naive way.
- Explain why file sync moves blocks, not files, and work out what an edit actually uploads.
- Split the design into a metadata store and a block store, and justify each one.
- Handle two devices editing at once, and edits made offline.
Heavy Systems + Your Rehearsal