Taming Noisy Neighbours

When one tenant slows everyone

In a pool, a single tenant can consume an unfair share of a resource: a bulk import saturating database I/O, an integration hammering the API in a loop, or one giant report hogging the worker queue. Everyone else sees slow pages, which is the noisy neighbour problem. Defences work at several layers. Rate limits per tenant (and per API key) cap request rates, often with a token bucket that allows short bursts. Quotas cap totals such as storage or monthly jobs, usually tied to the plan. Fair scheduling gives each tenant its own queue or a weighted share of workers, so one tenant's backlog of ten thousand jobs does not sit in front of another tenant's single job. Workload isolation moves heavy, predictable work such as analytics to read replicas or a warehouse. Finally, persistent heavy tenants are candidates for their own shard or silo. All of this needs per-tenant metrics to spot who is noisy.

A per-tenant token bucket

Each tenant gets its own bucket sized by plan. Time is passed in explicitly so the output is deterministic.

class TokenBucket:
    def __init__(self, rate_per_sec, capacity):
        self.rate, self.capacity = rate_per_sec, capacity
        self.tokens, self.last = capacity, 0.0

    def allow(self, now):
        # Refill based on elapsed time, never above capacity
        self.tokens = min(self.capacity, self.tokens + (now - self.last) * self.rate)
        self.last = now
        if self.tokens >= 1:
            self.tokens -= 1
            return True
        return False

LIMITS = {"free": (1, 2), "pro": (5, 10)}   # (refill/sec, burst)
buckets = {}

def allow(tenant, plan, now):
    if tenant not in buckets:
        buckets[tenant] = TokenBucket(*LIMITS[plan])
    return buckets[tenant].allow(now)

# zen (free) fires 4 requests at t=0; acme (pro) fires 4 too
print("zen ", [allow("zen", "free", 0.0) for _ in range(4)])
print("acme", [allow("acme", "pro", 0.0) for _ in range(4)])

Output:

zen  [True, True, False, False]
acme [True, True, True, True]

Water tanker for the society

When a housing society shares one water tank, each flat gets a fixed number of buckets per hour. A flat that wants to fill a swimming pool cannot drain the tank for everyone else; it waits for its share to refill, or pays for its own tanker.

Limit by tenant, not only by IP

IP-based limits fail for SaaS: many users of one company share an office IP, and one tenant's integration may run from many cloud IPs. Key limits on tenant and API key, and return HTTP 429 with a Retry-After header so well-behaved clients back off.

Quick check: A shared job queue processes jobs in strict arrival order. Tenant A enqueues 50,000 jobs; tenant B then enqueues one. What change best protects tenant B?

• Increase the queue's maximum length
• Process jobs in reverse order
• Delete tenant A's jobs
• Schedule fairly across tenants, for example per-tenant queues served round-robin