# Sizing a Cache — Cost & Capacity Estimation

Source: https://www.skillbyai.com/en/capacity-estimation/ce-cache-sizing

> Estimate cache memory from the hot working set and predict how hit ratio reduces backend load.

## Cache the working set, not everything

A cache is effective because access is usually **skewed**: a small fraction of items receives most of the reads. The **working set** is the data accessed within some window, such as a day. A common starting assumption is that a modest fraction of objects, say 20%, serves most requests, so the cache needs memory for that hot fraction, not the full dataset. Size it as **hot items x (value size + key size + per-item overhead)**; in-memory stores add metadata per entry, so do not assume raw value bytes alone. The payoff is measured by **hit ratio**. Backend load equals total reads x (1 - hit ratio), so moving from 90% to 99% hits cuts database reads by 10x. That non-linear benefit is why the last few percent of hit ratio are worth fighting for. Also plan for cache failure: if the cache restarts empty, the database briefly sees the full read load.

## Where the reads go

Client reads hit the cache first; only misses fall through to the database, so hit ratio directly controls database load.

![Three connected stages: client reads, cache, database.](assets/figures/capacity-estimation/section-6-map.svg) — Figure 6.1: client reads -> cache -> database

## Memory and backend load

Estimating cache memory and database read load for different hit ratios.

```python
total_items = 200_000_000
hot_fraction = 0.2
value_bytes, key_bytes, overhead_bytes = 1_000, 40, 60
read_qps = 50_000

cache_gb = total_items * hot_fraction * (value_bytes + key_bytes + overhead_bytes) / 1e9
print(f"cache memory needed: {cache_gb:.0f} GB")
for hit in (0.80, 0.90, 0.99):
    print(f"hit {hit:.0%}: DB reads {read_qps * (1 - hit):,.0f}/s")
```

Output:

```
cache memory needed: 44 GB
hit 80%: DB reads 10,000/s
hit 90%: DB reads 5,000/s
hit 99%: DB reads 500/s
```

## Plan for a cold cache

After a cache flush or a new deploy, every read misses. Protect the database with request coalescing, gradual warm-up or rate limits, or the cache restart becomes an outage.

**Quiz:** Reads are 20,000 per second. If the cache hit ratio rises from 90% to 98%, what happens to database reads?

- [x] They drop from 2,000/s to 400/s
- [ ] They drop from 18,000/s to 19,600/s
- [ ] They stay at 2,000/s
- [ ] They drop from 2,000/s to 1,600/s

*Answer:* They drop from 2,000/s to 400/s. Misses are 10% then 2% of 20,000: 2,000/s falls to 400/s, a 5x reduction from an 8-point change in hit ratio.
