# Resilience in the Service Mesh and Gateway — Rate Limiting, Circuit Breakers & Resilience Patterns

Source: https://www.skillbyai.com/en/resilience-patterns/l-mesh

> Configure timeouts, retries, outlier detection and connection limits in Envoy or Istio.

## Policies outside the code

A **service mesh** or gateway can apply resilience policies uniformly without changing application code. **Envoy**, the proxy beneath Istio and many gateways, supports per-route **timeouts** and **retry policies** (with retry conditions, per-try timeouts and retry budgets), **circuit breaking** via connection-pool limits (maximum connections, pending requests, concurrent requests and retries per upstream cluster), and **outlier detection**, which ejects individual unhealthy hosts from the load-balancing pool after consecutive errors and returns them after an ejection period. **Istio** exposes these through `VirtualService` (timeouts, retries, fault injection) and `DestinationRule` (connection pools, outlier detection). Mesh-level policies are excellent for consistent defaults, but the application still knows best about **idempotency**, **fallbacks** and **business priorities**, so combine both and avoid double retries (mesh plus library).

## Policies in the proxy beside each service

Sidecar or node proxies apply timeouts, retries and ejection between services.

![Pairs of boxes, each service box with a small proxy box attached, connected by arrows that pass only through the proxies.](assets/figures/resilience-patterns/section-7-map.svg) — Figure 7.1 — Resilience policies enforced by mesh proxies.

## Istio timeouts, retries and outlier detection

Retries are limited and per-try timeouts keep the total within the route timeout.

```yaml
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: payments
spec:
  hosts: [payments]
  http:
    - route:
        - destination: { host: payments }
      timeout: 2s
      retries:
        attempts: 2
        perTryTimeout: 800ms
        retryOn: connect-failure,refused-stream,unavailable,5xx
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: payments
spec:
  host: payments
  trafficPolicy:
    connectionPool:
      http:
        http1MaxPendingRequests: 100
        http2MaxRequests: 200
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 10s
      baseEjectionTime: 30s
      maxEjectionPercent: 50
```

## One retry layer only

If the mesh retries and the application library also retries, attempts multiply. Pick one place for retries per call path, usually the mesh for simple idempotent calls or the library when business logic decides.

**Quiz:** What does outlier detection in Envoy or Istio do?

- [x] Temporarily removes individual hosts that return repeated errors from the load-balancing pool
- [ ] Encrypts traffic between services
- [ ] Limits requests per user
- [ ] Caches responses at the edge

*Answer:* Temporarily removes individual hosts that return repeated errors from the load-balancing pool. Outlier detection ejects misbehaving hosts so traffic goes to healthy ones.