Caching Layers from Browser to Database

Caches at every hop

A request from a user can be answered by caches at many layers. The browser cache stores responses on the user's device according to HTTP headers; nothing is faster than not making a request. A CDN (content delivery network) caches responses at edge locations near users worldwide. A reverse proxy in front of your servers (NGINX, Varnish) can cache whole pages or API responses. Inside the application, an in-process cache (a hash map with eviction, such as Caffeine in Java) answers in nanoseconds but is per instance, while a distributed cache such as Redis or Memcached is shared by all instances. Finally, databases cache data pages and query plans in memory, and some workloads use materialised views. Each outer layer serves more users with less effort but is harder to invalidate precisely; inner layers are easier to control but every request still travels further.

Who answers the request?

The earlier a layer can answer, the less work everything behind it does.

browser cache     -> 0 network round trips; per user; controlled by Cache-Control / ETag
CDN edge          -> nearest PoP; shared by everyone in a region; purge by URL or tag
reverse proxy     -> in your datacentre; full pages / API responses
app in-process    -> nanoseconds; per instance; may diverge between instances
distributed cache -> ~sub-millisecond; shared by all instances; Redis / Memcached
database buffers  -> automatic; hot pages and indexes in RAM

Groceries at home, the corner shop and the wholesale market

Your fridge (browser) is instant, the corner shop (CDN edge) is a short walk, the supermarket (distributed cache) is a drive, and the wholesale market (database) is far away. You only drive to the market for what the closer places do not stock.

Quick check: Which cache layer is shared by all application instances but sits inside your own infrastructure?

• The browser cache
• An in-process hash map
• The CPU L1 cache
• A distributed cache such as Redis