Resilience in the Service Mesh and Gateway

Policies outside the code

A service mesh or gateway can apply resilience policies uniformly without changing application code. Envoy, the proxy beneath Istio and many gateways, supports per-route timeouts and retry policies (with retry conditions, per-try timeouts and retry budgets), circuit breaking via connection-pool limits (maximum connections, pending requests, concurrent requests and retries per upstream cluster), and outlier detection, which ejects individual unhealthy hosts from the load-balancing pool after consecutive errors and returns them after an ejection period. Istio exposes these through VirtualService (timeouts, retries, fault injection) and DestinationRule (connection pools, outlier detection). Mesh-level policies are excellent for consistent defaults, but the application still knows best about idempotency, fallbacks and business priorities, so combine both and avoid double retries (mesh plus library).

Policies in the proxy beside each service

Sidecar or node proxies apply timeouts, retries and ejection between services.

Istio timeouts, retries and outlier detection

Retries are limited and per-try timeouts keep the total within the route timeout.

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: payments
spec:
  hosts: [payments]
  http:
    - route:
        - destination: { host: payments }
      timeout: 2s
      retries:
        attempts: 2
        perTryTimeout: 800ms
        retryOn: connect-failure,refused-stream,unavailable,5xx
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: payments
spec:
  host: payments
  trafficPolicy:
    connectionPool:
      http:
        http1MaxPendingRequests: 100
        http2MaxRequests: 200
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 10s
      baseEjectionTime: 30s
      maxEjectionPercent: 50

One retry layer only

If the mesh retries and the application library also retries, attempts multiply. Pick one place for retries per call path, usually the mesh for simple idempotent calls or the library when business logic decides.

Quick check: What does outlier detection in Envoy or Istio do?

• Temporarily removes individual hosts that return repeated errors from the load-balancing pool
• Encrypts traffic between services
• Limits requests per user
• Caches responses at the edge