Cache Avalanche and Cache Penetration

Two more ways to overwhelm the database

A cache avalanche happens when a large share of keys become unavailable at once: many keys created together (after a deploy or a bulk warm-up) expire together, or the whole cache node restarts empty. Suddenly the database receives the full read load. Defences: TTL jitter so expiries spread out, staggered warm-up, cache replication and persistence so a node failure does not empty the cache, and circuit breakers or rate limits in front of the database so overload degrades service rather than causing collapse. Cache penetration happens when requests ask for keys that do not exist in the database, for example random IDs from a bot or a bug; every request misses and hits the database. Defences: negative caching with short TTLs, input validation (reject malformed IDs early), and a Bloom filter of existing keys that answers "definitely not present" in memory, with a small false-positive rate.

A Bloom filter guard

Requests for IDs that certainly do not exist never reach the cache or database.

from pybloom_live import ScalableBloomFilter   # any Bloom filter library works

known_ids = ScalableBloomFilter(error_rate=0.001)
for product_id in db.all_product_ids():           # rebuild periodically, add on create
    known_ids.add(product_id)

def get_product_guarded(product_id):
    if product_id not in known_ids:
        return None          # definitely absent: no cache or DB lookup
    return get_product(product_id)   # may still be absent (false positive), handled normally

A guard with a guest list

A doorman with a guest list turns away strangers instantly instead of sending each one inside to check every room. Occasionally the list has a similar name and someone gets in to check, but most pointless trips are avoided.

Quick check: Bots request millions of random, non-existent product IDs, and each one hits the database. What is this called and how can it be mitigated?

• Cache avalanche; add more replicas
• Cache penetration; use negative caching, validation and a Bloom filter
• Cache stampede; increase TTL
• Write-behind; flush faster