Rate Limiting, Circuit Breakers & Resilience Patterns

What you'll learn

• Explain how slow dependencies and uncontrolled retries cause cascading failures.
• Set deadlines and timeouts and design safe retries with backoff, jitter, budgets and idempotency keys.
• Configure circuit breakers, bulkheads, fallbacks and kill switches with common libraries.
• Implement and compare rate-limiting algorithms, including distributed limits in Redis.
• Protect services from overload with load shedding, adaptive concurrency limits, back-pressure and client-side throttling.
• Apply resilience policies in meshes and gateways, test them with fault injection and monitor them in production.

Syllabus

Why Resilience Patterns Exist

1. Failure Is Normal in Distributed Systems
2. A Map of the Patterns
3. Deadlines and Timeout Budgets

Timeouts and Retries

1. Choosing Timeouts
2. Retries, Backoff and Jitter
3. Retry Amplification, Retry Budgets and Hedging

Circuit Breakers

1. How a Circuit Breaker Works
2. Circuit Breakers in Resilience4j and Polly
3. Designing Breakers and Combining Patterns

Isolation and Graceful Degradation

1. Bulkheads
2. Fallbacks and Graceful Degradation
3. Kill Switches and Feature Flags for Resilience

Rate Limiting

1. Why and Where to Rate Limit
2. Rate Limiting Algorithms
3. Distributed Rate Limiting

Overload Protection

1. Load Shedding
2. Concurrency Limits and Back-Pressure
3. Client-Side Throttling and Cooperative Clients

Resilience in Libraries, Meshes and APIs

1. Resilience in the Service Mesh and Gateway
2. Idempotency Keys for Safe Retries
3. Choosing and Standardising Resilience Libraries

Testing, Observing and Revision

1. Testing Resilience with Fault Injection
2. Observing Resilience Patterns
3. Case Study: Hardening a Checkout Flow
4. Revision and Interview Questions