Core idea
A single server might handle anywhere from hundreds to tens of thousands of requests/second, depending on the work per request, before CPU, memory, or I/O saturates. Scale is not about being big — it’s about which resource becomes the bottleneck first and designing around it.
Mental model
Think of a restaurant. One chef can handle 20 covers. 200 covers? You need more chefs (horizontal scale), a bigger kitchen (vertical scale), or a smarter menu (algorithmic improvement). Every system design decision maps to one of these three moves.
Why it matters
- Every other topic in system design exists because of this problem
- Understanding bottlenecks tells you WHERE to apply which solution
- Interviewers want to see you identify the constraint before jumping to solutions
What to know
- CPU-bound: computation is the limit → parallelize, cache results
- Memory-bound: data doesn’t fit → offload to disk or distributed cache
- I/O-bound: disk/network is the limit → async, batching, read replicas
- Network-bound: too much data transferred → compression, CDN, edge caching
Common traps
- Premature optimization — don’t add complexity before you need it
- Fixing the wrong bottleneck — measure first, optimize second
- Ignoring the network — it is always slower than you assume
Recognise it
Recognition trigger: Any question about ‘millions of users’ or ‘high traffic’ → immediately ask: which resource will run out first? That shapes your entire design.
Practice question
How would you handle 10x traffic growth on your current system?
Apply at work
Answer these for the system you work on.
Go deeper
- Chapter 1: Reliable, Scalable and Maintainable Applications · Designing Data-Intensive Applications (Kleppmann), book
- Handling Overload (opens in a new tab) · Google SRE book