Upstream Groups and Balancing Methods

Spreading requests across servers

An upstream block names a group of servers that proxy_pass can target. The default method is round robin, optionally weighted (weight=3 sends three times as many requests to that server). least_conn sends each request to the server with the fewest active connections, better when request durations vary. ip_hash pins clients to servers by IP address (a simple form of stickiness, unreliable when many users share an IP). hash $key consistent routes by any key, such as $request_uri for cache-friendly distribution, using consistent hashing so adding a server moves few keys. NGINX Open Source also offers random two least_conn (the power-of-two-choices method). Server parameters include max_fails and fail_timeout (passive health checking), backup (used only when primaries are down) and down (temporarily removed). Prefer stateless application servers over session stickiness, so any server can handle any request.

Balancing methods

Round robin rotates, least connections picks the least busy, hashing pins a key to a server.

Weighted, least-connections and hashed upstreams

Choose a method per backend type.

upstream web_app {
    least_conn;
    server 10.0.1.11:8080 weight=2;       # bigger machine
    server 10.0.1.12:8080;
    server 10.0.1.13:8080;
    server 10.0.1.20:8080 backup;         # only when the others are unavailable
    keepalive 64;
}

upstream image_cache {
    hash $request_uri consistent;          # same image -> same cache node
    server 10.0.2.21:8080;
    server 10.0.2.22:8080;
    server 10.0.2.23:8080;
}

Stickiness hides statefulness

If your app only works with ip_hash, sessions are stored in server memory. Move them to a shared store (Redis, a database, signed cookies) so deploys and failures do not log users out.

Quick check: Requests to an API vary from 5 ms to several seconds. Which method usually balances better than plain round robin?

• ip_hash
• backup
• least_conn
• down