03.5 Rate limiting

Junior · Component

Core idea

Rate limiting controls how many requests a client can make in a time window. It prevents abuse, protects downstream services, and ensures fair usage. Without it, a single bad actor can take down your entire system.

Mental model

A nightclub with a bouncer: only 100 people allowed inside at once. New arrivals queue. Regular members get priority (tiered rate limits). Known troublemakers are turned away immediately (IP blocklist). The bouncer doesn’t care what you’re doing inside — just manages the door.

Why it matters

  • DDoS attacks, scrapers, and buggy clients can overwhelm unprotected APIs
  • Rate limiting is a contract: ‘you get N calls per second, use them wisely’
  • Senior engineers implement rate limiting at the infrastructure level, not in application code

What to know

  • Token bucket: bucket refills at rate R. Each request consumes a token. Allows bursts up to bucket size. Most common.
  • Leaky bucket: requests processed at fixed rate. Queue absorbs bursts but adds latency.
  • Fixed window counter: count resets every N seconds. Boundary attack: 200 req at 59s + 200 req at 61s = 400 in 2s.
  • Sliding window log: track exact timestamps of requests. Accurate but memory-heavy.
  • Sliding window counter: approximation using two windows. Good tradeoff.
  • Distributed rate limiting: use Redis with atomic incr/expire — share counter across API nodes

Common traps

  • Per-server rate limiting — one server says ‘you hit your limit’ while others accept requests
  • Not returning proper 429 Too Many Requests with Retry-After header — clients can’t adapt
  • Rate limiting by IP only — shared IPs (NAT) punish many users for one bad actor; use user ID

Recognise it

Recognition trigger: Any public API → rate limiting is mandatory. Ask: per IP or per user? What are the tiers? Where is the limit enforced (gateway level, not app level)? What algorithm (token bucket for APIs, leaky bucket for smooth downstream flow)?

Practice question

How would you implement a rate limiter for an API that allows 100 requests per minute per user, across 50 API servers?

Go deeper