Message Queues & Pub/Sub

Lesson · Junior

Decouple producers from consumers.

Why queues

A producer puts messages on a queue and moves on; consumers process them at their own pace. This smooths traffic spikes, isolates failures and lets you scale consumers independently. Examples: SQS, RabbitMQ, Kafka.

Queue vs pub/sub

Queue: each message is handled by one consumer (work distribution). Pub/sub: each message is delivered to every subscriber (event broadcast). Kafka does both via consumer groups and a replayable log.

Delivery guarantees

At-most-once may lose messages; at-least-once may duplicate them; exactly-once is hard. Practical rule: make consumers idempotent so duplicates are harmless, and use dead-letter queues for poison messages.