Rate Limits and Spend Caps

Per user, per feature, per day

Usage-based pricing means a bug, an abusive user or a viral moment can create a large bill quickly. Set rate limits per user and per account, spend caps per feature per day with alerts at thresholds (for example 50%, 80%, 100%), and provider-side budget limits. When a cap is reached, degrade gracefully to the fallback rather than failing silently. Review caps as the rollout grows.

A daily spend cap with fallback, run

I ran this with Python 3 (scipy 1.18.1 where imported) on example numbers, not data from a real product. With a $50 daily cap and a mix of normal and long requests, 4,733 requests are served before the cap is reached and the remaining 3,267 get the fallback. An 80% alert would have fired at roughly request 3,787.

class SpendCap:
    def __init__(self, daily_limit):
        self.limit, self.spent = daily_limit, 0.0
    def allow(self, est_cost):
        if self.spent + est_cost > self.limit:
            return False
        self.spent += est_cost; return True
cap = SpendCap(daily_limit=50.00)
allowed = blocked = 0
for i in range(8000):
    cost = 0.0084 if i % 10 else 0.0300          # every tenth request is a long one
    if cap.allow(cost): allowed += 1
    else: blocked += 1
print(f"allowed {allowed} requests, spent ${cap.spent:.2f} of ${cap.limit:.2f}; {blocked} served by fallback")
print("alert ops at 80%: would have fired at about request", int(0.8 * cap.limit / (0.9 * 0.0084 + 0.1 * 0.03)))

Output:

allowed 4733 requests, spent $50.00 of $50.00; 3267 served by fallback
alert ops at 80%: would have fired at about request 3787

Alert before the cap, not at it

An alert at 80% gives people time to decide whether to raise the cap or investigate a spike.

Quick check: What should happen when an AI feature reaches its daily spend cap?

• Crash the app
• Charge users more automatically
• Serve the fallback experience and alert the owners
• Silently keep spending