Limits: Slow Subscribers, Reconnects and Scale

Where Pub/Sub breaks down

Because Redis pushes messages to subscribers as fast as they are published, a slow subscriber builds up an output buffer inside Redis. To protect itself, Redis applies client-output-buffer-limit pubsub (by default a hard limit of 32 MB, or 8 MB sustained for 60 seconds) and disconnects subscribers that exceed it, so they lose messages. Reconnects create gaps: any message published between disconnect and re-subscribe is lost, so clients that need continuity should resynchronise state after reconnecting (for example, reload from the database or read recent entries from a stream). Every published message is sent to every subscriber, so cost grows with subscribers × message rate, and in a classic Redis Cluster a PUBLISH is broadcast to every node, which limits scalability; sharded Pub/Sub (next section) fixes that. Finally, there is no history: a new subscriber cannot ask what happened before it joined.

Watching and tuning output buffers

Look for subscribers with large output buffers before Redis disconnects them.

# list connected clients; check omem (output memory) and flags for subscribers
redis-cli CLIENT LIST TYPE pubsub

# default limit for pub/sub clients: hard 32mb, soft 8mb for 60 s
redis-cli CONFIG GET client-output-buffer-limit

# raising limits only hides a consumer that cannot keep up;
# better: make subscribers fast (hand work to a local queue) or move to Streams

Do slow work off the subscriber loop

A subscriber callback that writes to a database or calls an API for each message will fall behind under load. Push messages into an in-process queue and process them in worker threads, or use Streams with consumer groups.

Quick check: What does Redis do to a Pub/Sub subscriber whose output buffer exceeds the configured limit?

• Disconnects the subscriber, which then misses messages
• Pauses publishers
• Writes the messages to disk
• Converts the channel to a stream