03.1 Load balancers

Intern · Component

Core idea

A load balancer distributes incoming traffic across multiple servers. It prevents any single server from becoming a bottleneck and provides redundancy — if one server dies, traffic routes to others.

Mental model

Think of a toll plaza with 10 booths. Without a traffic officer, all cars pile into the first visible booth. The officer (load balancer) waves cars to the shortest queue. Remove one booth — the officer routes around it. Add a booth — it automatically joins the pool.

Why it matters

  • Single server = single point of failure
  • Single server = hard ceiling on throughput
  • Load balancers enable horizontal scaling — the primary scaling strategy

What to know

  • L4 (transport layer): routes by IP/port, very fast, no content inspection — use for raw TCP/UDP
  • L7 (application layer): routes by HTTP headers, URL, cookies — enables content-based routing
  • Algorithms: round-robin (simple), least-connections (smart), IP hash (sticky), weighted round-robin
  • Health checks: LB pings servers every N seconds, removes unhealthy ones
  • Sticky sessions: same user always goes to same server — needed for stateful apps (but avoid statefullness)

Common traps

  • Load balancer itself becomes SPOF — always deploy in pairs with failover
  • Sticky sessions defeat load balancing — prefer stateless services with external session store
  • L7 LB adds latency — use L4 where content routing isn’t needed

Recognise it

Recognition trigger: Multiple servers exist → there’s a load balancer in front. L7 for HTTP/microservices routing. L4 for game servers, raw TCP. If interviewer asks ‘what if one server goes down?’ — health checks + LB is your answer.

Practice question

How does a load balancer know a server is unhealthy? What happens to in-flight requests when a server is removed?

Go deeper