Publisher Confirms and Returns

Did the broker really get it?

basic_publish is fire-and-forget: it returns before the broker has stored anything. If the connection drops, the message may be lost without any error. Publisher confirms fix this: after confirm_delivery() (confirm.select in the protocol), the broker sends an ack for each message once it has been handled, which for persistent messages on durable or quorum queues means safely written (to disk, or replicated to a quorum of nodes). A nack means the broker could not take responsibility, and the publisher should retry. Confirms can be awaited per message (simple but slow), in batches, or asynchronously with callbacks, which gives the best throughput. Separately, a message that matches no queue is silently dropped unless you publish with mandatory=True, in which case the broker returns it to the publisher. Combine confirms, mandatory publishing and retries with the same message_id for reliable publishing.

Publish, store, confirm

The publisher keeps the message until the broker confirms it has been safely stored.

Confirms and mandatory publishing with pika

An unroutable or unconfirmed message raises an error the publisher can handle.

channel.confirm_delivery()          # enable publisher confirms on this channel

try:
    channel.basic_publish(
        exchange="orders",
        routing_key="order.placed.in",
        body=payload,
        properties=pika.BasicProperties(
            delivery_mode=pika.DeliveryMode.Persistent,
            message_id=event_id,
            content_type="application/json",
        ),
        mandatory=True,             # return if no queue matches
    )
except pika.exceptions.UnroutableError:
    log.error("no queue bound for order.placed.in")
except pika.exceptions.NackError:
    schedule_retry(event_id)        # broker refused responsibility; retry with same message_id

Use the outbox for business events

Confirms tell you the broker has the message, but not that it matches your database. For events tied to database changes, write them to an outbox table in the same transaction and publish from there with confirms.

Quick check: What does a publisher confirm (ack) from the broker tell the publisher?

• A consumer has processed the message
• The message was routed to every queue in the vhost
• The broker has taken responsibility for the message (for example, persisted or replicated it)
• The message has expired