# Queues, Load Levelling and Back-Pressure — Scalability, Availability & Reliability

Source: https://www.skillbyai.com/en/scalability/p-async

> Absorb bursts with queues and protect systems with back-pressure and batching.

## Turning spikes into steady work

Not every task must finish before the response. Putting work on a **queue** (sending email, resizing images, generating invoices) lets the web tier answer quickly while **workers** process at a steady rate: the queue absorbs the spike, a pattern called **load levelling**. Workers scale on queue depth, and a slow downstream system no longer slows users. Queues are not infinite, though. When producers outpace consumers for long periods, the backlog and latency grow without limit, so systems need **back-pressure**: signals that make producers slow down, such as bounded queues that reject or block, HTTP 429/503 with `Retry-After`, or flow control in streaming protocols. **Batching** improves throughput by amortising per-request overhead: one database insert of 500 rows instead of 500 inserts, at the cost of a little latency.

## A bounded buffer that applies back-pressure

When the buffer is full, the producer is told to back off instead of growing memory forever.

```python
import asyncio

queue: asyncio.Queue = asyncio.Queue(maxsize=1000)   # bounded

async def accept_upload(job):
    try:
        queue.put_nowait(job)
        return 202, "accepted"
    except asyncio.QueueFull:
        return 503, "busy, retry later"            # back-pressure to the client

async def worker():
    while True:
        batch = [await queue.get()]
        while len(batch) < 100 and not queue.empty():
            batch.append(queue.get_nowait())        # batch for efficiency
        await save_thumbnails(batch)
        for _ in batch:
            queue.task_done()
```

## A dam on a river

A dam stores the monsoon flood and releases it steadily through the turbines. But if it keeps raining for weeks, you must open the spillways (back-pressure) or the dam fails.

**Quiz:** What does back-pressure achieve in an overloaded pipeline?

- [ ] It makes the queue infinitely large
- [x] It signals producers to slow down so the backlog and memory stay bounded
- [ ] It deletes the oldest data silently
- [ ] It disables batching

*Answer:* It signals producers to slow down so the backlog and memory stay bounded. Back-pressure propagates overload upstream instead of letting queues grow without limit.