# Concurrency, Throttling and Downstream Protection — Serverless & Edge

Source: https://www.skillbyai.com/en/serverless-edge/se-concurrency

> Estimate required concurrency, explain throttling, and use concurrency limits to protect fragile dependencies.

## Scaling your code is easy; scaling everything else is not

In the standard Lambda model each environment handles one request at a time, so the **concurrency** you need is roughly requests per second multiplied by average duration in seconds (Little's law). Each account has a per-region concurrency quota shared by all its functions; once it is exhausted, new invocations are **throttled**. Synchronous callers receive a 429 error, while asynchronous and queue-based events are retried later. Two settings help. **Reserved concurrency** guarantees a function a slice of the quota and also caps it at that number, which is a cheap way to protect a fragile downstream such as a small database or a partner API with rate limits. For SQS triggers, the event source mapping's **maximum concurrency** setting limits how many concurrent invocations that queue can drive. The broader lesson: serverless scales your code almost instantly, but nothing else in your system automatically scales with it, so put limits and queues in front of the slow parts.

## Little's law in three lines

Doubling duration doubles the concurrency you need just as much as doubling traffic does. Slow dependencies are a scaling problem, not only a latency problem.

```python
# Little's law: concurrency = arrival rate x time each request spends in the system
def concurrency_needed(requests_per_sec: float, avg_duration_ms: float) -> float:
    return requests_per_sec * (avg_duration_ms / 1000)

for rps, ms in [(200, 50), (200, 800), (2000, 800)]:
    print(f"{rps:>5} req/s x {ms:>3} ms -> {concurrency_needed(rps, ms):6.0f} concurrent executions")
```

Output:

```
  200 req/s x  50 ms ->     10 concurrent executions
  200 req/s x 800 ms ->    160 concurrent executions
 2000 req/s x 800 ms ->   1600 concurrent executions
```

## Toll plaza booths

If each car takes 30 seconds at a booth and 20 cars arrive per minute, you need at least 10 booths open. Cars arriving faster than booths can serve them form a queue, and if the road beyond the plaza is narrow, opening more booths only moves the jam further down.

**Quiz:** A function receives 300 requests per second and each takes 400 ms on average. Roughly how many concurrent executions does it need?

- [x] 120
- [ ] 750
- [ ] 300
- [ ] 12

*Answer:* 120. Concurrency is about rate x duration = 300 x 0.4 s = 120.
