Rate Limiting

Token bucket and friends

Rate limiters cap requests per client, API key or IP. The token bucket refills at a steady rate and allows bursts up to its capacity; the leaky bucket smooths output; fixed and sliding windows count requests per period. In a distributed system, counters usually live in a shared store such as Redis with atomic operations. Rejected requests get 429 Too Many Requests with a Retry-After header.

A token bucket allowing bursts, run

I ran this with Python 3.12.3 using only the standard library; inputs are fixed or seeded, so the output is reproducible. A bucket of 10 tokens refilling at 5 per second allows a burst of 10 at once, then 5 more after a second, and a steady 5 requests per second indefinitely.

# Token bucket: 5 requests/second sustained, bursts of up to 10
class TokenBucket:
    def __init__(self, rate, capacity):
        self.rate, self.capacity = rate, capacity
        self.tokens, self.last = capacity, 0.0
    def allow(self, now):
        self.tokens = min(self.capacity, self.tokens + (now - self.last) * self.rate)
        self.last = now
        if self.tokens >= 1:
            self.tokens -= 1
            return True
        return False

b = TokenBucket(rate=5, capacity=10)
burst = [b.allow(0.0) for _ in range(12)]          # 12 requests at t=0
print("burst at t=0  :", burst.count(True), "allowed,", burst.count(False), "rejected")
later = [b.allow(1.0) for _ in range(7)]           # 1s later: 5 tokens refilled
print("7 more at t=1 :", later.count(True), "allowed,", later.count(False), "rejected")
steady = [b.allow(2.0 + i * 0.2) for i in range(10)]   # exactly 5 req/s
print("steady 5 req/s:", steady.count(True), "of 10 allowed")

Output:

burst at t=0  : 10 allowed, 2 rejected
7 more at t=1 : 5 allowed, 2 rejected
steady 5 req/s: 10 of 10 allowed

Limit at several layers

Edge limits stop abuse cheaply; per-service limits protect databases and downstream dependencies.

त्वरित जाँच: Which HTTP status signals rate limiting?

• 500 Internal Server Error
• 404 Not Found
• 429 Too Many Requests
• 301 Moved Permanently