What Caching Buys You

Keep a copy closer to where it is needed

A cache stores a copy of data in a place that is faster or closer than the original source, so repeated requests skip the expensive work. The speed differences are enormous: reading main memory takes on the order of 100 nanoseconds, a round trip to a cache server in the same datacentre roughly half a millisecond, a typical database query a few milliseconds or more, and a request across continents 100 to 150 ms. Caching cuts latency for users, reduces load on databases and services (often the main reason), absorbs traffic spikes, and reduces cost such as database capacity and egress bandwidth. Good candidates are read often, change rarely, are expensive to compute or fetch, and can tolerate being slightly stale: product pages, configuration, user profiles, rendered fragments, static assets. Poor candidates change constantly, are read once, or must always be exactly current, such as account balances during a payment.

The cost of going further away

Each step away from the CPU costs orders of magnitude more time, which is why caches pay off.

Is this data worth caching?

Quick questions to ask before adding a cache.

question                                         good sign for caching
-----------------------------------------------  ---------------------------------
how often is it read compared with written?      reads >> writes (e.g. 100:1)
how expensive is producing it?                   slow query, remote call, heavy render
how stale may it be?                             seconds to hours are acceptable
how big is the hot set?                          fits in memory (often a small % of all data)
is it the same for many users?                   shared data caches far better than per-user
what if the cache is empty?                      origin can survive the load (plan for it)

A cache is a copy, so it can be wrong

Every cache trades freshness for speed. Before adding one, decide how stale the data may be and how changes reach the cache. Those two answers shape everything else.

Quick check: Which data is the best candidate for caching?

• Product descriptions read thousands of times per minute and edited a few times a day
• A wallet balance during payment authorisation
• A one-time password
• A log line written once and never read