Pub/Sub

Asynchronous messaging at global scale

Pub/Sub is a fully managed, global messaging service. Publishers send messages to a topic; each subscription on that topic receives its own copy of every message, so several services can react independently. Subscriptions are pull (your workers fetch messages) or push (Pub/Sub sends HTTPS requests to an endpoint such as Cloud Run), and there are export subscriptions that write directly to BigQuery or Cloud Storage. A subscriber must acknowledge each message before the ack deadline; otherwise it is redelivered. Delivery is at least once by default, so handlers must tolerate duplicates; pull subscriptions can enable exactly-once delivery. Ordering keys keep messages with the same key in order, and a dead-letter topic catches messages that fail too many times. Unacknowledged messages are retained for seven days by default.

A pull subscriber that acknowledges only on success

Raising before ack() lets Pub/Sub redeliver the message later.

from google.cloud import pubsub_v1

subscriber = pubsub_v1.SubscriberClient()
path = subscriber.subscription_path("shop-dev-123456", "orders-worker")

def callback(message):
    try:
        process_order(message.data, order_key=message.attributes.get("orderId"))
        message.ack()
    except Exception:
        message.nack()  # redeliver soon; after max attempts it goes to the dead-letter topic

future = subscriber.subscribe(path, callback=callback)
future.result()  # block and process messages

# setup:
# gcloud pubsub topics create orders
# gcloud pubsub subscriptions create orders-worker --topic=orders --ack-deadline=60 \
#   --dead-letter-topic=orders-dlq --max-delivery-attempts=5

A newspaper with many subscribers

The publisher prints one edition (topic), and every subscriber gets their own copy. If a reader does not sign for the delivery (ack), the courier comes back and tries again.

Quick check: Two independent services must each receive every order event. How do you configure Pub/Sub?

• One topic with one shared subscription
• One topic with a separate subscription per service
• Two topics with no subscriptions
• One subscription with an ordering key