# Long-lived Connections and Load Balancers — WebSockets & Server-Sent Events

Source: https://www.skillbyai.com/en/realtime-web/rt-scaling-connections

> Explain what each open connection costs and how load balancers, sticky sessions and limits affect real-time servers.

## Connections are the unit of cost

A REST server's load is measured in requests per second; a real-time server's load is mostly **concurrent connections**. Each open WebSocket or SSE stream holds a TCP socket, a file descriptor, kernel buffers and your per-connection state, even when idle. A well-written event-loop server can hold tens of thousands of mostly idle connections per process, but limits appear early: the OS file-descriptor limit (`ulimit -n`, often 1024 by default for a shell), memory per connection, and timeouts on every proxy in the path. **Load balancers** must support the Upgrade header and long-lived connections. Because a connection stays on the server that accepted it, balancing happens only when clients connect, so a freshly added server receives no existing users until they reconnect. **Sticky sessions** matter for Socket.IO's long-polling fallback and for any protocol where consecutive HTTP requests must reach the same process; pure WebSockets do not need them, because one connection never moves.

## Where a broadcast travels at scale

Clients reach one of many app servers through a load balancer. Servers publish to a shared broker, which fans the message out to every server holding interested clients.

![Four connected stages in order: clients, load balancer, multiple app servers, pub/sub broker.](assets/figures/realtime-web/section-7-map.svg) — Figure 7.1: clients -> load balancer -> app servers -> pub/sub broker

## Phone lines, not letters

A REST API is a post office: letters arrive, get handled and are done. A real-time server is a call centre: each caller holds a line for as long as they like. Capacity planning is about how many lines you can keep open, not how many letters per second you can sort.

## Raise limits deliberately and measure

Before load-testing, raise the process file-descriptor limit (for example `LimitNOFILE` in a systemd unit), set proxy read timeouts above your heartbeat interval, and track open connections as a metric. Load-test with idle connections plus realistic message rates, not just a burst of messages.

**Quiz:** You add a fourth server behind the load balancer of a busy WebSocket app. Why does it receive almost no traffic at first?

- [ ] Load balancers cannot route WebSockets to new servers
- [ ] WebSockets require sticky sessions to be reset manually
- [ ] The new server must be restarted first
- [x] Existing connections stay on their servers; only new or reconnecting clients reach the new one

*Answer:* Existing connections stay on their servers; only new or reconnecting clients reach the new one. Balancing happens at connect time. Long-lived connections do not migrate, so new capacity fills only as clients reconnect.
