Timeouts, Retries, Circuit Breakers and Load Shedding

Containing the blast radius

A small set of patterns prevents most cascading failures. Timeouts on every remote call, with each layer's timeout shorter than its caller's. Retries only for transient errors and idempotent operations, with exponential backoff and jitter and a cap. Circuit breakers stop calling a dependency that is failing, fail fast for a cool-off period, then probe with a few trial requests before closing again. Bulkheads isolate resources (separate connection pools or thread pools per dependency) so one bad dependency cannot consume everything. Load shedding rejects excess requests early (HTTP 503 or 429) rather than accepting work that will time out anyway, preferably dropping low-priority traffic first. Graceful degradation serves a reduced experience, such as cached prices or a page without recommendations, instead of an error. The next course in this catalogue covers these patterns in depth.

Backoff with full jitter

Spreading retries randomly prevents clients from retrying in lockstep.

import random, time

TRANSIENT = (TimeoutError, ConnectionError)

def call_with_retry(fn, attempts=4, base=0.1, cap=2.0):
    for attempt in range(attempts):
        try:
            return fn()
        except TRANSIENT:
            if attempt == attempts - 1:
                raise
            sleep_for = random.uniform(0, min(cap, base * 2 ** attempt))   # full jitter
            time.sleep(sleep_for)

Electrical fuses and room circuits

A circuit breaker trips before the whole house wiring burns; separate circuits for the kitchen and bedrooms (bulkheads) mean a faulty kettle does not plunge the bedrooms into darkness.

त्वरित जाँच: What does a circuit breaker do when a dependency keeps failing?

• Retries faster
• Stops calling it for a while and fails fast, then probes to see if it recovered
• Restarts the dependency
• Increases the timeout