Quorum Queues

Replicated with Raft

A quorum queue replicates its contents across several nodes (typically 3 or 5) using the Raft consensus algorithm. One replica is the leader; others are followers. A message is confirmed to the publisher only after a majority of replicas have written it, so the queue survives the loss of a minority of nodes without losing confirmed messages, and a new leader is elected automatically. Quorum queues are always durable, are declared with x-queue-type: quorum, and are the recommended default for data safety. They track delivery counts and enforce a delivery limit (default 20 in 4.0), dead-lettering poison messages that keep failing instead of looping forever, and support at-least-once dead-lettering to avoid losing messages on their way to the DLX. Trade-offs: higher latency and disk use than a non-replicated queue, and they are designed for data that is consumed reasonably promptly, not for multi-million message backlogs held for days.

Declaring a quorum queue with sensible settings

Replication, delivery limit and dead-lettering in one declaration.

channel.queue_declare(
    queue="payments.capture",
    durable=True,
    arguments={
        "x-queue-type": "quorum",
        "x-quorum-initial-group-size": 3,         # replicas
        "x-delivery-limit": 5,                     # poison messages dead-letter after 5 attempts
        "x-dead-letter-exchange": "payments.dlx",
        "x-dead-letter-strategy": "at-least-once", # safer dead-lettering
        "x-overflow": "reject-publish",            # required for at-least-once dead-lettering
    },
)

A committee that votes

A quorum queue is a committee of three clerks. A decision (message) is recorded only when at least two have written it in their books. If one clerk falls ill, the other two carry on and nothing confirmed is lost.

त्वरित जाँच: When does a quorum queue confirm a published message?

• As soon as the leader receives it in memory
• After every consumer has acknowledged it
• Only after the queue is empty
• After a majority of replicas have written it