Designing for Scale · Requirements to Architecture

The problem this post solves

Most bad architectures are not bad because someone chose the wrong database. They are bad because the design started before anyone agreed on what was being built, for how many people, with what tolerance for staleness, and at what cost of being wrong.

"Design a system to share short notes" is not a specification. It is a prompt. Between that prompt and a defensible architecture sits a step that is routinely skipped: converting an ambiguous sentence into a bounded problem with numbers attached. This post is about that step, and about why drawing boxes first is the most expensive habit in system design.

The claim is simple. The architecture is a consequence of the constraints. If the constraints are not written down, the architecture is a guess wearing a diagram.

First principles

Start from what a system actually is: a machine that moves data between people and storage, under limits.

There are exactly two families of requirement, and conflating them is the most common early error.

Functional requirements describe what the system does. They are verbs. A user creates a note. A user retrieves a note by link. A note expires. You can write a test for each one, and the test passes or fails.

Non-functional requirements describe the conditions under which it must keep doing those things. They are adjectives and numbers. Ten thousand reads per second. Under two hundred milliseconds at the ninety-ninth percentile. Available when one region is on fire. Durable enough that losing a note is a serious incident rather than an annoyance.

Functional requirements determine what components exist. Non-functional requirements determine what shape they take. A note store and a note store that serves a hundred thousand reads per second across three continents contain the same nouns and almost none of the same architecture.

The core model

The four-step frame below is not a trick. It is the same skeleton mature design documents follow, compressed into something that fits on a whiteboard.

The dotted lines matter as much as the solid ones. Discovering during capacity estimation that a feature implies a petabyte of hot storage is not a failure of the process; it is the process working. The cheapest place to discover an impossible requirement is immediately after writing it down.

Step one in practice: bounding the problem

Three questions convert a prompt into a problem.

Who uses it, and how many are there? Total registered users is nearly useless. Daily active users is the number that generates load. The ratio between them varies enormously by product and is the single most common place where an estimate goes wrong by an order of magnitude.

What do they do, and in what proportion? Every system has a read/write ratio, and it dictates the architecture more than almost anything else. A read-heavy system wants caching, read replicas and denormalisation. A write-heavy system wants partitioning, batching and back-pressure. A system where both are high wants a much larger budget.

What must never happen? This is the question people skip, and it is where durability and consistency requirements live. Losing a draft is bad. Losing a payment is unacceptable. Showing a stale follower count is fine. Showing a stale account balance is a regulatory event. The severity of the worst case sets the floor for how much machinery is justified.

Writing requirements that constrain

A requirement that cannot be violated is not a requirement. Compare:

"The system should be fast." This constrains nothing. Every design satisfies it, so it cannot be used to reject any design.

"Reads return in under 200 ms at p99, measured at the edge, for the top 90 percent of keys by traffic." This rejects designs. It rules out a cold read path that crosses an ocean. It implies a cache. It admits that the long tail is allowed to be slower, which is what makes it affordable.

The second version does three things the first does not: it names a percentile rather than an average, it names a measurement point, and it scopes itself to a subset of traffic. Each of those is a place where a vague target quietly becomes a much more expensive one.

Percentiles deserve particular care. An average latency target is nearly meaningless for user experience, because a page that makes twenty backend calls will hit the tail on most requests. If each call independently meets a p99 of 200 ms, the probability that a twenty-call page avoids the tail entirely is roughly 0.99 to the twentieth power — about 82 percent. Put differently, nearly one page load in five hits at least one slow call. Tail latency compounds; averages hide it.

Capacity and constraints

Capacity estimation earns its place only when a number changes a decision. An estimate that decorates the design without ruling anything out is wasted time, and reviewers can tell.

The useful outputs are few. Requests per second, derived from daily actions divided by seconds in a day, then multiplied by a peak factor because traffic is never flat. Storage growth per year, derived from writes per day times bytes per write times retention. Bandwidth, derived from reads per second times response size. Working set, meaning the portion of data that is hot enough to justify keeping in memory.

Each of those has a decision hanging off it. Requests per second decides whether one machine can serve the traffic. Storage growth decides whether the data fits on one node or must be partitioned. Bandwidth decides whether a content delivery network is a nice-to-have or load-bearing. Working set decides the cache tier size, and therefore much of the cost.

The arithmetic should stay symbolic and round. Precision is false comfort here: the input assumptions carry far more error than the multiplication does. What matters is the order of magnitude and the decision it forces.

Architecture

Only now do the boxes appear, and they appear as answers to the constraints rather than as a remembered template.

Every element in that diagram should be traceable to a requirement. The edge exists because of a latency target and a geographic spread of users. The cache exists because of a read/write ratio and a working set that fits in memory. The queue exists because some work was identified as tolerating delay, which is a statement about requirements, not about technology. The replicas exist because of an availability target and a read volume that exceeds one node.

If a component cannot be traced back to a constraint, it is decoration, and it should be removed. Extra components are not free: each one adds a failure mode, an operational burden and a consistency question.

The reverse check is equally important. Walk the non-functional list and ask which part of the diagram satisfies each entry. A durability requirement with no replication or backup answering it is a hole. An availability requirement with a single point of failure in the path is a hole. Holes found at the whiteboard cost nothing.

Choosing the deep dive

A design review has time for one component to be examined properly. Choosing which one is itself a judged decision, and the right choice is the component where the design is most likely to be wrong — usually the one under the most extreme constraint, or the one where a trade-off was asserted without justification.

The wrong choice is the component the designer happens to find most interesting.

Failure modes

Designing from a remembered template. Producing the same diagram regardless of the prompt is the clearest signal that requirements were never engaged with. The tell is a component that cannot be justified when questioned.

Averages instead of percentiles. A design that meets an average latency target can still be slow for most users of a multi-call page, because tail latency compounds across calls.

Confusing registered users with active users. This single substitution routinely produces estimates that are wrong by one or two orders of magnitude, in whichever direction is least convenient.

Treating peak as average. Traffic is not flat. Designing to the daily mean guarantees the system is under-provisioned at exactly the moment it matters most.

Accepting an unbounded requirement. "All historical data must be instantly queryable" and "the system must never lose a write" are not requirements until someone attaches a retention window and a durability target. Unbounded requirements silently import unbounded cost.

Deferring the consistency question. Whether a read may return stale data is a product decision that determines replication strategy, cache invalidation and multi-region topology. Leaving it implicit means it gets decided by accident, usually by whichever component was built first.

Trade-offs that matter

The recurring tension in this phase is between bounding the problem tightly enough to design against, and bounding it so tightly that the design cannot survive a plausible change in the product.

Over-constraining produces a system exquisitely fitted to today's numbers that requires rearchitecting when one assumption shifts. Under-constraining produces a system with no defensible choices in it at all, which is worse: at least the over-fitted system works today.

The practical resolution is to state which assumptions are load-bearing. If the design depends on the read/write ratio staying above roughly one hundred to one, say so explicitly. That sentence turns a hidden fragility into a monitored one, and it tells whoever inherits the system which number to watch.

Similarly, requirements should record what was deliberately excluded. A design that says "cross-region strong consistency is out of scope; regional reads may be up to a few seconds stale" is stronger than one that quietly omits the question, because the omission is now a decision someone can revisit rather than a landmine someone will discover.

Four worked cases

The frame is only worth something if it survives contact with different prompts. What follows are four prompts run through the same four steps. The point is not the answers; it is that the same procedure produces different architectures, because the constraints differ.

Case A — Short-link service

Requirements. Functional: create a short code for a URL, resolve a code to a URL, expire a code, count resolutions. Non-functional: resolution p99 under 50 ms at the edge, creation p99 under 300 ms, resolution must survive a region loss, a lost creation is an annoyance, a wrong resolution is an incident.

Numbers. Say 1M creations/day and 500M resolutions/day. That is roughly 12 writes/s and 5,800 reads/s average, call it 3x at peak, so ~17,000 reads/s. Read/write ratio ≈ 500:1. Payload is tiny — 500 bytes/row, so 1M/day is 500 MB/day, 180 GB/year. The whole hot set is a few GB.

What the numbers decide. A 500:1 ratio and a working set that fits in RAM means the cache is load-bearing, not decorative. 180 GB/year means a single primary node is fine for years — do not shard. Resolution surviving a region loss plus a tiny dataset means full replication everywhere beats partitioning. "A wrong resolution is an incident" means codes must never be reused, which kills random generation without a uniqueness check and pushes toward a monotonic counter with base-62 encoding.

Case B — Group chat

Requirements. Functional: send a message to a room, receive messages in near-real-time, fetch history, show delivery state. Non-functional: delivery p95 under 200 ms end-to-end, a delivered message must never be lost, ordering must be consistent per room, history may be seconds stale.

Numbers. 10M DAU, 40 messages sent/user/day = 400M writes/day ≈ 4,600 writes/s average, ~14,000 at peak. But fan-out is the real number: average room size 8 means ~110,000 deliveries/s at peak. Storage: 400M × 1 KB = 400 GB/day, 146 TB/year.

What the numbers decide. Writes are now comparable to reads — the cache stops being the story and the write path becomes it. 146 TB/year forces partitioning on day one, and "ordering consistent per room" makes room_id the obvious partition key. Persistent connections at 10M DAU means a connection tier separate from the logic tier, because those scale on different axes (memory vs CPU). "Never lost" means acknowledge only after durable commit, never after enqueue.

Case C — News feed

Requirements. Functional: post, follow, render a ranked feed. Non-functional: feed p95 under 300 ms, feed may be up to a minute stale, a dropped post is bad but not an incident.

Numbers. 100M DAU, 20 feed loads/day = 2B reads/day ≈ 23,000/s, ~70,000 peak. Writes: 5M posts/day ≈ 58/s. Ratio ≈ 400:1 — but with fan-out, since one post by a user with 10M followers touches 10M feeds.

What the numbers decide. The staleness allowance of one minute is the single most valuable requirement in the list: it makes precomputation legal. Fan-out-on-write becomes viable for ordinary users, but the celebrity tail breaks it — 10M writes for one post. So the architecture is hybrid: precompute for the median user, compute-at-read for the high-fan-out tail. That hybrid is not a clever trick; it falls directly out of the follower distribution being heavy-tailed.

Case D — Metering and billing

Requirements. Functional: ingest usage events, aggregate per account per period, produce an invoice. Non-functional: an event must never be double-counted and never dropped, aggregates may lag by minutes, invoices must be reproducible years later.

Numbers. 2B events/day ≈ 23,000/s, 70,000 peak. 200 bytes/event = 400 GB/day raw.

What the numbers decide. "Never double-counted" dominates everything else. It forces idempotency keys on ingest, exactly-once semantics at the aggregation boundary, and an immutable raw event log as the system of record. "Reproducible years later" forbids destructive aggregation — you keep the raw log and recompute, rather than trusting a running total. Notice that the QPS here is similar to Case C but the architecture shares almost nothing with it, because the correctness requirement differs.

The requirement that decides the shape

Across those four, one requirement in each did most of the work — and in no case was it the QPS number. Cases C and D have nearly identical throughput and almost nothing in common structurally.

This is worth stating as a rule: throughput decides how many machines; correctness and staleness decide what the machines do. Designers who lead with QPS end up with a well-sized version of the wrong system.

The staleness allowance in particular behaves like a currency. A system permitted one minute of staleness can precompute, cache aggressively, replicate asynchronously and batch its writes. A system permitted zero staleness gives up all four. When a stakeholder says "it must be real-time," the correct response is to ask what breaks at one second, then ten, then sixty — the answer is usually a number, and that number is worth more to the design than any other sentence in the room.

Tracing the boxes back

The reverse check deserves a picture, because it is the step most often skipped. Every component below is annotated with the requirement it exists to satisfy. Delete the requirement and the component must go with it.

Numbers worth memorising

Estimation is only fast if the constants are already in your head. These are the ones that actually change decisions, rounded hard on purpose.

Time. 86,400 seconds in a day — round to 100,000 and your QPS estimate is 15% low, which is inside the error of every other assumption you made. 1M requests/day ≈ 12/s. 100M/day ≈ 1,200/s. 1B/day ≈ 12,000/s.

Peak factor. Consumer traffic peaks at roughly 2–3x the daily mean; a single-timezone product can hit 5x. Say which you are assuming.

Latency floors. Memory reference ~100 ns. SSD random read ~100 µs. Same-datacenter round trip ~0.5 ms. Cross-continent round trip ~150 ms — that last one is a hard floor set by the speed of light in fibre, and no amount of engineering removes it. If the requirement is 50 ms globally and the data lives in one region, the requirement is already impossible and someone should say so in the first five minutes.

Storage. 1 KB × 1M/day = 1 GB/day ≈ 365 GB/year. A single modern node comfortably holds single-digit TB. So roughly: under ~5 TB/year, do not shard.

Tail compounding. A page that makes 10 independent calls, each with a p99 of 100 ms, has roughly a 10% chance of at least one slow call — so the page p90 is what the call p99 was. This is why per-service p99s do not add up to a good user experience, and why fan-out width is itself a latency decision.

Where this frame breaks

It would be dishonest to present this as universal. It is a frame for systems whose difficulty is scale, and there are systems whose difficulty is something else.

Regulated and correctness-first systems. In payments, ledgers and clinical systems the binding constraint is auditability and invariants, not throughput. A ledger doing 50 transactions per second can still be the hardest system in the building. Starting from QPS there is a category error; start from the invariants that must never be violated and design the audit trail first.

Latency-floor systems. In trading or real-time control, the requirement is a hard ceiling on the tail, not a percentile of a distribution — and the architecture becomes about removing hops, not adding them. Most of the toolkit above (queues, caches, replicas) adds variance and is therefore disqualified.

Systems where the requirement is genuinely unknown. For a product still finding its shape, over-specifying the non-functional requirements is a way of pretending to knowledge you do not have. The right move is a deliberately simple design plus written triggers: "we shard when the primary passes 2 TB", "we split the connection tier when we pass 200k concurrent". That converts an unknown into a monitored threshold rather than a guess.

Small systems. If the honest answer is that one machine serves the traffic with room to spare, the correct design is one machine, and the mark of seniority is saying so rather than producing a distributed system to look thorough.

Companion deep report

A longer working paper accompanies this post: full arithmetic for all four cases, the constants table, a six-dimension self-grading rubric, a catalogued failure-mode list, and a five-problem timed drill set.

What to carry forward

The frame is small enough to hold without notes: bound the problem, quantify it, design against the quantities, then examine the weakest component properly.

The discipline underneath it is smaller still. Before drawing anything, write down the numbers the drawing has to satisfy. After drawing anything, check that every box traces back to one of them. Most design failures are a violation of one of those two sentences.

Capacity estimation, which is where the numbers actually come from, is involved enough to deserve its own treatment — including how to keep the arithmetic honest and how to tell an estimate that constrains from one that merely impresses.