Series · 20 parts
Designing for Scale — Requirements to Review
System design in the order the decisions actually get made: constraints before boxes, numbers before technology choices. Foundations in parts 1–4, canonical designs in 5–10, data and ML systems in 11–18, and two full design reviews to close.
- 1Designing for Scale · Requirements to ArchitectureFrom a one-line prompt to a defensible architecture: separating functional from non-functional requirements, quantifying who/what/how-many, and refusing to draw a single box until the constraints are on the board.
- 2Designing for Scale · Estimation That ConstrainsBack-of-envelope arithmetic is only worth doing when a number rules something out. How to derive requests per second, storage growth, bandwidth and working set — and how to tell a constraining estimate from a decorative one.
- 3Designing for Scale · The Storage Decision TreeDatabases are not religions. Choosing between relational, wide-column, document, and blob storage by mapping the access pattern before looking at the engine.
- 4Designing for Scale · CAP and the Consistency SpectrumWhy strong consistency is a latency penalty you choose to pay, why eventual consistency is not a defect, and how to use the CAP theorem to end distributed systems arguments.
- 5Designing for Scale · URL ShortenerThe classic system design starting point. It looks like a toy problem until you have to guarantee collision-free generation at scale while keeping read latency under ten milliseconds.
- 6Designing for Scale · Distributed Rate LimiterHow to stop abuse without stopping legitimate traffic. An engineering breakdown of Token Bucket, Leaky Bucket, Sliding Window algorithms, and how to execute them across a fleet of servers without locking Redis.
- 7Designing for Scale · Chat and Real-time CommunicationWhy HTTP fails for real-time delivery, how WebSockets change the load balancer math, and the architecture required to deliver a message to a million users simultaneously.
- 8Designing for Scale · Newsfeed and Fan-outWhy reading from a database to render a feed is too slow, and how the fan-out-on-write model precomputes millions of feeds in memory before users even ask for them.
- 9Designing for Scale · Typeahead and Search AutocompleteHow to return query suggestions in under fifty milliseconds while the user is still typing, using Tries, offline aggregations, and edge caching.
- 10Designing for Scale · Notification SystemsWhy sending a push notification is not a fire-and-forget API call. Handling rate limits, provider outages, deduplication, and the retry queues required to make delivery reliable.
- 11Designing for Scale · Real-Time AnalyticsHow to count billions of events in real-time without crushing your database, using stream processing, time-window aggregations, and Lambda architecture.
- 12Designing for Scale · Change Data Capture (CDC)Why dual-writes fail, how the transaction log is the only true source of state, and the architecture required to stream database changes to search indexes and caches without losing data.
- 13Designing for Scale · The Data LakehouseWhy the data warehouse and data lake converged. Moving from expensive, proprietary compute-storage monoliths to open table formats like Iceberg, Hudi, and Delta Lake.
- 14Designing for Scale · Metering and BillingWhy billing systems cannot drop a single event, the difference between at-least-once and exactly-once processing, and how to build idempotent pipelines that survive crashes without double-charging users.
- 15Designing for Scale · The Feature StoreBridging the gap between data engineering and machine learning. How to serve features for model training offline, and serve those exact same features for inference in five milliseconds online.
- 16Designing for Scale · Model ServingWhy wrapping a PyTorch model in a Flask API is a prototype, not a production system. Handling GPU saturation, dynamic batching, and the difference between CPU and GPU scaling.
- 17Designing for Scale · RAG and Vector SearchWhy LLMs hallucinate, how Retrieval-Augmented Generation grounds them in reality, and the architecture required to execute semantic search over millions of documents in milliseconds.
- 18Designing for Scale · LLM-as-a-Judge and Release GatesHow to ship generative AI to production without shipping a liability. Breaking subjective prompts into deterministic grading rubrics, automated evaluation pipelines, and multimodal scoring.
- 19Designing for Scale · End-to-End Design Review IPulling the components together. How to trace a requirement from population estimates through capacity math to a defensible caching strategy, using a live design review as the frame.
- 20Designing for Scale · End-to-End Design Review IIA second live design review, tearing apart an event ticketing system. Handling massive, instantaneous write spikes, distributed transactions, and inventory locks without destroying the database.