Read Models, Projections and Eventual Consistency

Keeping read models up to date

A projection is code that listens to events (or change-data-capture records) and updates a read model: insert a row on OrderPlaced, update the status on OrderShipped, increment a counter on ProductViewed. One stream of events can feed many projections: a customer order list, a warehouse picking view, a search index, a revenue dashboard. Projections must be idempotent and able to be rebuilt from scratch, by replaying events into a fresh table, which also lets you add new read models later. The cost is eventual consistency: just after a command succeeds, the read model may not show the change yet. Handle it in the UI: return the new ID and show the result optimistically, poll or push an update, display "processing", or read from the write model for the user's own just-created item.

A projection for the customer orders view

Upserts make it safe to replay or receive duplicates.

def project(event, db):
    t, d = event["type"], event["data"]
    if t == "OrderPlaced":
        db.execute(
            "INSERT INTO customer_orders_view (order_id, customer_id, placed_at, item_summary, status) "
            "VALUES (%s, %s, %s, %s, 'PLACED') ON CONFLICT (order_id) DO NOTHING",
            (d["orderId"], d["customerId"], event["time"], summarise(d["items"])),
        )
    elif t == "OrderShipped":
        db.execute(
            "UPDATE customer_orders_view SET status = 'SHIPPED', eta = %s WHERE order_id = %s",
            (d.get("eta"), d["orderId"]),
        )
    elif t == "OrderCancelled":
        db.execute(
            "UPDATE customer_orders_view SET status = 'CANCELLED' WHERE order_id = %s",
            (d["orderId"],),
        )

The railway departure board

The departure board is a read model: it is updated from the control room's decisions, usually within seconds. Occasionally a train has already left before the board catches up, so the station staff plan for that delay.

Quick check: A user creates an order and is immediately redirected to "My orders", but the new order is missing. What is the most likely cause?

• The command failed silently
• The database lost the write
• The schema registry rejected the event
• The read model has not processed the event yet (eventual consistency)