Rate Limiting in Practice: The Algorithm Matters Less Than What You Return

The client hit your limit. Your server returned a bare 429, no explanation, no hint about when to come back.

So the client did the only thing it could: retried immediately. And again. And again — because nothing in your response told it not to.

You now have a rate limiter that generates more traffic than it prevents.

The Part the Standard Refuses To Answer

429 Too Many Requests comes from RFC 6585 (Nottingham and Fielding, April 2012). The definition is short:

“The 429 status code indicates that the user has sent too many requests in a given amount of time (‘rate limiting’).”

Then comes the sentence that quietly hands you the whole problem:

“Note that this specification does not define how the origin server identifies the user, nor how it counts requests.”

Both hard questions, left open on purpose. Who is being limited, and how you count — those are yours to answer, and everything below follows from them.

Four Algorithms, and Where Each One Breaks

Fixed window

Count requests per calendar interval. At the top of each minute, reset to zero.

It is one counter and one expiry per key, which is why everyone starts here. It also silently permits double your limit.

A client that spends its entire allowance in the last second of one window, then spends it again in the first second of the next, has sent twice the limit inside a two-second span — and every request was within the rules.

If your limit is 100 per minute, a determined client delivers 200 in a rolling minute. Your capacity planning said 100.

Sliding window log

Store a timestamp per request, drop everything older than the window, count what remains.

Exact. Also the most expensive thing on this page: memory grows with request volume, not with the number of clients. A single busy key can hold thousands of timestamps, and every check walks them.

Correct, and rarely worth it.

Sliding window counter

Keep the current window’s count and the previous window’s, then weight the old one by how far into the current window you are. Roughly seventy percent through a minute, you count thirty percent of the previous minute’s requests.

An approximation, and a good one. Constant memory, no boundary cliff. It can be slightly wrong when traffic is extremely bursty, in exchange for two integers per key.

Token bucket

A bucket holds tokens up to a maximum. Tokens refill at a constant rate. Each request takes one; an empty bucket means rejection.

This is the default worth reaching for. It permits a burst up to the bucket size — which matters, because real clients are bursty and a limiter that punishes normal behaviour is a support ticket — while holding the long-run average at the refill rate.

Storage is two values: token count and last refill time. You compute the refill lazily on read rather than running a timer.

Leaky bucket

Requests enter a queue that drains at a fixed rate. Overflow is rejected.

Where token bucket allows bursts, leaky bucket flattens them completely — output is constant no matter what arrives. Use it when the thing downstream genuinely cannot take a spike: a payment processor, a legacy system, an external API with its own limits.

The Race That Shows Up the Moment You Scale Out

One server, one in-memory counter — fine. Add a second instance and the counter has to move to shared storage. That is where the interesting bug lives:

# BROKEN — do not ship this
count = redis.get(key)
if count and int(count) >= LIMIT:
    return 429
redis.incr(key)          # ← several instances can all be here
handle_request()

Between the read and the increment there is a window. Four instances handling simultaneous requests all read 99, all conclude there is room, and all proceed. Your limit of 100 just admitted 103.

The fix is the same shape as the one in idempotency: make the decision one atomic operation instead of a check followed by an act.

-- INCR returns the new value, so the increment IS the check
local current = redis.call('INCR', KEYS[1])
if current == 1 then
  redis.call('EXPIRE', KEYS[1], ARGV[1])
end
return current

INCR returns the post-increment value, so incrementing is the check — there is no gap to lose. Set the expiry only when the counter is created, or a busy key never expires and its window never resets.

Two decisions to make deliberately:

What happens when Redis is down? Fail open and an outage in your limiter becomes unlimited traffic to everything behind it. Fail closed and a limiter outage takes down the API. Neither is universally right, but the choice must be conscious and written down.

Every request now costs a network round trip. For high-volume tiers, a common pattern is a local allowance checked in-process with periodic reconciliation against shared state — approximate, much faster, and usually the right trade at the edge.

What To Actually Return

The algorithm decides whether to allow the request. The response decides whether the client behaves well afterwards. The second one matters more, and gets far less attention.

The status code

429 is the intended code. RFC 6585 asks for two things:

“The response representations SHOULD include details explaining the condition, and MAY include a Retry-After header indicating how long to wait before making a new request.”

Note the strength: explanation is SHOULD, Retry-After is only MAY. In practice, send both. A 429 with no Retry-After tells a client that it failed but not what to do, and the default behaviour of most HTTP clients in that situation is to try again immediately.

Also: 429 responses MUST NOT be cached. The RFC is explicit. A cached 429 keeps rejecting a client whose quota reset minutes ago.

The headers nobody has right

Here is the part that surprises people.

X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset are not a standard. They never were. They are a convention that spread by copying, and because nothing specified them, implementations disagree about what they mean.

The IETF draft that aims to fix this documents the damage in its own appendix. X-RateLimit-Remaining is used in the wild to mean:

• seconds remaining until the window resets
• milliseconds remaining until the window resets
• a UNIX timestamp
• a formatted date

Four incompatible meanings for one header name. And the names themselves fork — X-RateLimit-Limit versus X-Rate-Limit-Limit.

So what does the actual proposal look like? Not like that at all. draft-ietf-httpapi-ratelimit-headers is active — version 11, updated May 2026 — and it defines two fields using HTTP structured field syntax:

RateLimit-Policy: "burst";q=100;w=60,"daily";q=1000;w=86400
RateLimit: "default";r=50;t=30

RateLimit-Policy advertises the quota policy and stays stable across responses: q is the quota, w the window in seconds. RateLimit reports the live state: r is remaining quota, t the effective window. A server can advertise several policies at once — a burst limit and a daily limit — and name which one it is reporting on.

Three details in that design worth stealing even if you never adopt the header:

Seconds, not timestamps. The draft uses delay-seconds deliberately: timestamps require clock synchronisation between client and server, and clock skew or adjustment quietly breaks them.

Quota is not only requests. The spec defines three units — requests, content-bytes and concurrent-requests. The same mechanism limits bandwidth and concurrency, not just call counts.

Partition keys. A pk parameter tells the client which bucket the numbers refer to — per user, per application, per resource. Without it, a client acting for several users cannot tell whose quota it is seeing.

One honest caveat before you build on it: the draft is a work in progress, its syntax has been reworked between versions, and its most recent HTTP directorate review came back not ready. Use the ideas; wait on the wire format.

Machine-readable reasons

The draft also registers three problem types (RFC 9457), which finally make why you were throttled something a client can branch on:

The middle one is the useful novelty. “You are over quota” and “we are shedding load” are completely different situations that have historically shared a status code, leaving clients unable to tell a personal limit from a system-wide brownout.

The Security Part Most Articles Skip

A polite 429 is still work. RFC 6585’s security considerations are blunt:

“When a server is under attack or just receiving a very large number of requests from a single party, responding to each with a 429 status code will consume resources. Therefore, servers are not required to use the 429 status code; when limiting resource usage, it may be more appropriate to just drop connections.”

Rate limiting is fairness between legitimate clients. It is not DDoS protection. Answering a flood courteously means doing parsing, routing and serialisation work on the attacker’s behalf, several million times.

Your limiter can become an oracle. If failed authentication attempts consume quota, an attacker can probe an endpoint and learn about another user’s traffic from the quota that remains. Decide explicitly whether 401 and 403 count — and if they do, make sure the partition key cannot be chosen by the caller.

Honesty causes stampedes. If your quota resets at a fixed time and you truthfully tell every client when, every client returns at that exact instant. The draft names the problem and the fix in one line: add jitter to the window you advertise.

Limits are hints, not enforcement. From the draft’s own security section: throttling does not prevent clients from issuing requests. Headers help well-behaved clients cooperate. They do nothing about the others, and the server still needs mechanisms that stop resource exhaustion regardless.

What Good Looks Like

• The limiting key is deliberate — per user, per API key, per IP — and documented, because clients cannot cooperate with a rule they can’t see
• Token bucket unless you have a specific reason to want something else
• The check-and-increment is atomic; no read-then-write
• Redis failure behaviour is a decision on paper, not an accident of the code path
• Every 429 carries an explanation and a Retry-After
• 429 responses are never cached
• Advertised reset times carry jitter
• Whether 4xx responses consume quota is decided, not inherited
• Something cheaper than your application handles abuse at the edge

The Bottom Line

Choosing between token bucket and sliding window will occupy an afternoon. Which one you pick will rarely be the thing that matters.

What matters is the reply. A client that receives a clear limit, a remaining count and a time to come back will pace itself and stop hammering you. A client that receives a bare 429 will retry in a loop, and your limiter will have manufactured the very load it was installed to prevent.

Rate limiting is a conversation. Most implementations only hold up one end of it.

That same principle — that the failure response is part of the design, not an afterthought — runs through sending a million notifications without falling over, where the retry policy shapes the whole system.