Lesson 3 / 25

Choosing Between Pub/Sub, Lists, Streams and a Broker

Decide when Redis messaging is enough and when a dedicated broker fits better.

Good enough, or the wrong tool?

Redis messaging is a great fit when you already run Redis, messages are small, volumes fit in memory, and you want low latency with few moving parts: live notifications to WebSocket servers, cache invalidation across instances, lightweight job queues, short-lived event streams between a handful of services, and activity feeds. Prefer a dedicated broker when you need long retention on disk at large scale (Kafka), rich routing with exchanges, per-message TTL, priorities and dead-letter exchanges (RabbitMQ), strong durability guarantees on every write, or fully managed queues with no servers (SQS, Pub/Sub, Service Bus). Remember that Redis keeps data in memory, persistence is configurable rather than guaranteed per write, and replication is asynchronous, so a failover can lose recently acknowledged data. Choose deliberately, and document the delivery guarantee you are relying on.

A decision table

Match the requirement to the primitive.

requirement                                            choose
-----------------------------------------------------  -------------------------------
live fan-out, loss acceptable (presence, typing, UI)   Pub/Sub (sharded in cluster)
invalidate local caches on all app instances           Pub/Sub
simple job queue, one consumer per job                 Streams + consumer group (or BLMOVE list)
every event must be processed, with retries            Streams + consumer group + XACK
replay recent history for a new consumer               Streams
weeks/months of retention, very high volume            Kafka (or similar log)
routing rules, priorities, DLX, per-message TTL        RabbitMQ
zero operations, cloud-native                          SQS / Pub/Sub / Service Bus

Write down the guarantee

"Notifications may be lost if a WebSocket node restarts" or "orders are processed at least once from a stream with at most a few seconds of loss on failover": stating it prevents surprises during incidents.

Quick check: Which requirement suggests a dedicated log such as Kafka rather than Redis Streams?

  • Low-latency fan-out to a few services
  • A small job queue
  • Months of retention of very high-volume events on disk
  • Cache invalidation messages
Answer

Months of retention of very high-volume events on disk — Redis keeps streams in memory, so very long, very large retention is better suited to disk-based logs.