Hot Keys and Big Keys

When one key is too popular or too large

Distributed caches spread keys across nodes, but a single hot key (a viral product, the home page configuration, a celebrity profile) always lives on one node, which can saturate its CPU or network while others idle. Fixes: a small local in-process cache (L1) in front of the distributed cache with a very short TTL, so most reads never leave the application instance; key replication, storing copies under several suffixes (home:config#1 … #8) and reading a random one; and moving truly static data to the CDN. Big keys (multi-megabyte values or huge lists and hashes) slow every operation on them, block single-threaded servers such as Redis while they are processed, and make network transfer and eviction expensive. Keep values small (kilobytes, not megabytes), split large collections into pages, compress where useful, and use Redis's --bigkeys or --memkeys scans and slow-log to find offenders.

Local L1 cache in front of Redis

A one-second local cache absorbs most reads of a hot key on each instance.

from cachetools import TTLCache

local = TTLCache(maxsize=10_000, ttl=1.0)    # tiny, short-lived, per instance

def get_config(name):
    if name in local:
        return local[name]                   # nanoseconds, no network
    value = redis.get(f"config:v1:{name}")
    if value is None:
        value = load_config_from_db(name)
        redis.set(f"config:v1:{name}", value, ex=300)
    local[name] = value
    return value

Find hot keys before they find you

Track the top keys by request count (many client libraries and proxies can sample this). A key receiving a large share of all traffic is a capacity risk even if the cluster looks healthy on average.

Quick check: Why does a single extremely hot key overload one cache node even in a large cluster?

• Each key is stored on one node (shard), so all its traffic goes to that node
• Clusters cannot store popular keys
• Hot keys are always big keys
• TTLs do not apply to hot keys