Choosing and Standardising Resilience Libraries

Make the safe path the default path

Resilience works best when every team gets good defaults without thinking. Common libraries: Java: Resilience4j (circuit breaker, retry, rate limiter, bulkhead, time limiter), Spring Cloud CircuitBreaker as an abstraction; .NET: Polly v8 and Microsoft.Extensions.Http.Resilience, whose standard handler adds rate limiting, total timeout, retry, circuit breaker and per-attempt timeout to HttpClient; Go: libraries such as sony/gobreaker and failsafe-go, plus context deadlines built into the language; Node.js: opossum and cockatiel; Python: tenacity for retries and pybreaker for breakers. A platform team can wrap these in a shared, versioned client template with sensible defaults (timeouts, retry rules, breaker thresholds, metrics), so services start safe and override only with reason. Pair library defaults with mesh policies and document which layer owns which concern.

A standard resilience handler for HttpClient (.NET)

One line adds a pre-configured pipeline; options can then be tuned per client.

builder.Services
    .AddHttpClient<PaymentsClient>(c => c.BaseAddress = new Uri("https://payments.internal"))
    .AddStandardResilienceHandler(options =>
    {
        options.AttemptTimeout.Timeout = TimeSpan.FromMilliseconds(800);
        options.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(3);
        options.Retry.MaxRetryAttempts = 2;
        options.CircuitBreaker.FailureRatio = 0.5;
        options.CircuitBreaker.MinimumThroughput = 20;
    });

Review defaults like code

A default retry count of 5 copied into fifty services can turn one dependency blip into an outage. Keep resilience defaults in one shared place, review changes carefully and roll them out gradually.

Quick check: Why might a platform team provide a shared HTTP client template?

• To force every service to use the same database
• To remove the need for monitoring
• So all services start with safe timeouts, retries, breakers and metrics by default
• To disable retries everywhere