Rate limiting hides two different questions: can the infrastructure safely accept more work, and is this user entitled to consume it? The InfoQ comparison of Uber and OpenAI is interesting because the companies optimize different parts of that decision.
Uber’s engineering account describes service-mesh clients enforcing probabilistic drops using directives from zone aggregators and regional controllers. Decisions stay local while aggregate information propagates through the control plane. Uber explicitly accepts a short feedback delay and describes failing open when the control plane is unavailable.
OpenAI’s account describes an access waterfall combining usage limits and credit balances, with synchronous access decisions and asynchronous settlement. Product entitlement and overload protection have different correctness requirements.
Approximate protection versus exact entitlement
Consider a hypothetical service receiving 15,000 requests per second against a 10,000-request target. A drop probability of one third yields an expected admitted rate of 10,000. It does not guarantee exactly 10,000 admissions in each window, and delayed measurements can miss a sudden burst.
That can be a reasonable capacity control. It would be a poor substitute for a balance ledger where spending the same credit twice is unacceptable. An access decision that consumes scarce entitlement needs a defined reservation or accounting boundary, not just a random drop.
Simplified Uber-style feedback loop. It protects aggregate capacity rather than implementing exact financial accounting.
Separate failure policies
For a service-protection limiter, continuing with a last known directive or failing open might preserve availability. For a financial entitlement check, the same behavior could permit spending without a valid balance. The policy must follow the resource being protected.
| Decision | Useful correctness question |
|---|---|
| Capacity shedding | How far may admitted traffic overshoot? |
| Tenant fairness | Over what interval is fairness measured? |
| Credit reservation | Can concurrent requests spend the same balance? |
| Settlement | Can duplicate events charge twice? |
A gateway may perform all four functions, but sharing a process does not make their invariants interchangeable.
Watch the feedback loop
For a proposed deployment, I would monitor directive age, admission rate, retry amplification, and downstream saturation together. When a control update lags, the operator needs to distinguish stale policy from a genuinely underprovisioned dependency.
A shadow rollout can record decisions before enforcing them. Test uneven caller sizes, regional failover, and a retrying client. The limiter succeeds when it protects useful throughput without creating a new dependency that fails with the service it was meant to protect.
Related reading: Overload control and useful work.
Archive note: added November 18, 2026, and filed under 2026-02 as part of the systems reading series.