# Latency Orders of Magnitude — Cost & Capacity Estimation

Source: https://www.skillbyai.com/en/capacity-estimation/ce-latency-numbers

> Recall the rough latency of memory, SSD, network and disk access and use the gaps to reason about design.

## The latency ladder

You do not need exact figures, only the **order of magnitude** and the gaps between rungs. Reading from a CPU cache takes around a nanosecond. Reading **main memory** takes on the order of 100 nanoseconds. A **random read from an NVMe SSD** is in the tens to around a hundred microseconds, roughly 100 to 1,000 times slower than memory. A **network round trip inside one data centre** is commonly in the range of a few hundred microseconds. A **spinning-disk seek** is several milliseconds. A **round trip between continents** is tens to a couple of hundred milliseconds, bounded by the speed of light in fibre. Each jump is roughly 10x to 1000x. These gaps explain classic design choices: cache hot data in memory, avoid many sequential network calls per request, and keep users close to servers with regional deployments or CDNs.

## Each rung is far slower than the last

Moving from CPU cache to main memory, SSD, the data-centre network and finally a cross-continent link, latency grows by roughly one to three orders of magnitude at each step.

![Five connected stages: CPU cache, main memory, SSD, data-centre network, cross-continent network.](assets/figures/capacity-estimation/section-2-map.svg) — Figure 2.1: CPU cache -> main memory -> SSD -> data-centre network -> cross-continent network

## How latency budgets add up

A request that makes sequential calls pays every latency in turn. Here we use rough, illustrative values.

```python
ns = 1e-9
us = 1e-6
ms = 1e-3
steps = {
    "10 cache-miss memory reads": 10 * 100 * ns,
    "3 SSD random reads": 3 * 100 * us,
    "5 sequential in-DC RPCs": 5 * 500 * us,
    "1 cross-continent round trip": 150 * ms,
}
for name, sec in steps.items():
    print(f"{name:30s} {sec * 1000:8.3f} ms")
print(f"{'total':30s} {sum(steps.values()) * 1000:8.3f} ms")
```

Output:

```
10 cache-miss memory reads        0.001 ms
3 SSD random reads                0.300 ms
5 sequential in-DC RPCs           2.500 ms
1 cross-continent round trip    150.000 ms
total                           152.801 ms
```

## Distance dominates

Notice that one cross-continent round trip outweighs everything else combined. When a user-facing request feels slow, first count how many times it crosses a long network path before tuning code.

**Quiz:** Which is closest to the latency of a random read from main memory?

- [x] About 100 nanoseconds
- [ ] About 100 microseconds
- [ ] About 10 milliseconds
- [ ] About 150 milliseconds

*Answer:* About 100 nanoseconds. Main memory access is on the order of 100 ns. SSD random reads are around tens to a hundred microseconds, disk seeks several milliseconds, and intercontinental round trips over 100 ms.
