Rate Limiting and Abuse Control
The endpoint was three lines long. Take an email address, look up the account, send a password-reset link. It had shipped eighteen months earlier and nobody had touched it, because it worked.
Then one Tuesday it sent 63,000 emails in about four minutes.
Someone had pointed a script at POST /auth/forgot-password with a list of scraped addresses. No login required, because that is the whole point of a reset endpoint. Each call did real work: a database read, a token write, and a handoff to the email provider that cost a fraction of a cent and, far worse, burned sender reputation with every message delivered to a stranger who never asked for it. By the time the on-call woke up, the provider had throttled the entire account and legitimate resets were bouncing.
Every public endpoint is a free compute button for a stranger. Some buttons are cheap and some cost you money, an email quota, or an LLM invoice, but all of them are pressable by anyone with curl. A rate limiter is the thing that says, politely and cheaply: you, specifically, have done enough for now.
The one question a limiter answers
Strip away the algorithms and a rate limiter does exactly one thing. On every request it asks: has this key done too much, too fast? If yes, reject it before the expensive work runs. If no, let it through and record that it happened.
Two decisions fall out of that sentence, and they are not equally important.
- What is “this key”? IP address, user id, API key, or something you compute. This choice decides who gets throttled and who gets caught in the blast radius. It matters more than anything else.
- What is “too much, too fast”? The algorithm and its numbers. This is where token buckets and sliding windows live, and it is genuinely interesting, but it is the second question, not the first.
Beginners obsess over the algorithm. People who have been paged obsess over the key. Let’s start where it counts.
What you key on
The key is the identity you count against. Get it wrong and the fanciest algorithm just throttles the wrong people.
IP address is the tempting default because it needs no login. It is also a blunt instrument. Carrier-grade NAT (CGNAT) puts thousands of unrelated mobile users behind a single public address, and a corporate office or a university can do the same. Set an IP limit low enough to stop an attacker and you lock out an entire building. Set it high enough to spare the building and a real attacker is nowhere near it. Worse, a botnet or a rented pool of proxies hands the attacker a fresh IP per request, so the one thing you were counting resets to zero every time.
User id is far better, but you only have it after authentication. So the honest design uses both: a coarse IP limit on the unauthenticated edge to keep anonymous floods manageable, and a tighter per-user limit once you know who is calling. For third-party traffic, the API key plays the user’s role.
Per endpoint beats one global number. A single limit across your whole API is almost always wrong. Reading a product page and resetting a password are not the same risk, and they should not share a budget. Give cheap read endpoints a generous limit and give the dangerous ones their own tight ones.
Which endpoints are dangerous? The ones that cost you something on the far side:
- Login and anything that verifies a secret, because unlimited attempts are a brute-force machine.
- Password reset, magic links, OTP and email or SMS send, because each one spends money and reputation, and because they are a favourite tool for mailbombing a victim.
- Signup, because free accounts are the raw material of abuse.
- Anything that calls an LLM, a paid third-party API, or runs a heavy query. Here a request is not just load, it is a line item on your bill.
The algorithms, and how each one fails
Now the second question. Every algorithm below decides “too much, too fast” differently, and every one has a failure mode you should be able to name before you pick it.
Fixed window
The simplest thing that works. Chop time into fixed slots (each minute, say), keep one counter per key per slot, increment on each request, reject once the counter passes the limit. When the clock ticks over to the next minute the key gets a fresh counter, which is just a new slot.
// naive, in-memory version: good for understanding, wrong for production
const hits = new Map();
function fixedWindow(key, limit, windowMs) {
const slot = Math.floor(Date.now() / windowMs);
const k = `${key}:${slot}`;
const n = (hits.get(k) || 0) + 1;
hits.set(k, n);
return n <= limit; // true = allow
}It is cheap: one counter, one number. It also has a famous hole. The counter resets on the slot boundary, but real time does not care about your boundary. A client that fires a full limit’s worth of requests in the last moment of one window and another full limit’s worth in the first moment of the next has sent double the limit in a couple of seconds, and each window is, on its own books, perfectly within bounds.
For login throttling on a per-account key this is often fine, because doubling “5 attempts per minute” to 10 for one second changes nothing. For anything where the burst itself is the danger, keep reading.
Sliding window log
Fix the boundary problem exactly: store the timestamp of every request, and on each new one, drop everything older than the window and count what remains. If the count is under the limit, record the new timestamp. There is no seam, because the window slides continuously with each request.
In Redis this is a sorted set, and the prune-count-add has to be one atomic step (more on why below):
-- KEYS[1] = key, ARGV = now(ms), windowMs, limit, member
redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, ARGV[1] - ARGV[2]) -- drop old
local used = redis.call('ZCARD', KEYS[1])
if used < tonumber(ARGV[3]) then
redis.call('ZADD', KEYS[1], ARGV[1], ARGV[4])
redis.call('PEXPIRE', KEYS[1], ARGV[2])
return 1
end
return 0It is precise, and it doubles as an audit trail of exact request times. The cost is memory: you store one entry per request per key. A limit of 10,000 per hour means up to 10,000 timestamps sitting in memory for every active client. On a high-traffic API with many keys, that adds up fast. Reach for it when accuracy is worth the RAM, such as a high-value payment or auth path, not as your default.
Sliding window counter
The practical compromise, and the one to reach for when you are not sure. Keep just two fixed-window counters, the current slot and the previous one, and estimate the rolling count by weighting the previous slot by how much of it still overlaps the window.
The formula is the whole idea:
estimate = previous_count × (1 − elapsed) + current_countwhere elapsed is how far you are into the current slot, from 0 to 1. A quarter of the way into this minute, the previous minute still overlaps three-quarters of your rolling window, so it counts at 0.75. Near the end of the minute it barely counts at all. The seam spike smooths out, and you paid for two integers instead of a timestamp per request.
It is an approximation. It assumes the previous window’s requests were spread evenly, so in rare cases it lets a hair more or less through than a perfect count would. For general-purpose API limiting that margin is noise, and the memory savings are enormous. This is a fine default.
Token bucket
The usual right answer when you want to allow bursts on purpose. Picture a bucket that holds up to capacity tokens and refills at a steady rate tokens per second. Each request spends a token. If the bucket has one, the request goes through; if it is empty, the request is rejected until enough time passes to drip a token back.
Two numbers give you two independent knobs. Capacity is how big a burst you tolerate. Rate is the sustained throughput you will grant over the long run. A capacity of 20 with a refill of 5 per second says: a client that has been quiet can fire 20 requests at once, then settles to 5 per second. That matches how real clients behave, quiet then bursty, far better than a flat limit.
The refill is lazy: you do not run a timer topping up buckets. You store the token count and the timestamp of the last touch, then on each request compute how many tokens should have dripped in since then and cap at capacity. One key, two fields.
-- token_bucket.lua KEYS[1] = bucket
-- ARGV = capacity, rate(per sec), now(sec, fractional), cost
local cap = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])
local b = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(b[1]) or cap
local ts = tonumber(b[2]) or now
tokens = math.min(cap, tokens + (now - ts) * rate) -- drip since last touch
local ok = 0
if tokens >= cost then
tokens = tokens - cost
ok = 1
end
redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('EXPIRE', KEYS[1], math.ceil(cap / rate) + 1) -- clean idle keys
return { ok, tokens }Calling it from Node is one round trip:
const [ok, left] = await redis.eval(TOKEN_BUCKET, {
keys: [`rl:user:${userId}:search`],
arguments: ['20', '5', String(Date.now() / 1000), '1'],
});
if (!ok) return tooManyRequests(reply, Math.ceil(1 / 5)); // retry after ~1 tokenLeaky bucket
The token bucket’s mirror image. Same bucket, but instead of spending tokens you pour requests in, and the bucket drains at a fixed rate through a hole in the bottom. If it overflows, requests spill (get rejected). The point of difference is the output: a leaky bucket emits a perfectly smooth, constant stream no matter how bursty the input, which is exactly what you want in front of a downstream that cannot handle spikes at all.
Counting when you have more than one server
Here is the detail that turns a textbook limiter into a real one. Your app does not run as a single process. It runs as N instances behind a load balancer, and a request can land on any of them. If each instance keeps its counter in a local Map, then each instance independently allows a full limit, and your real limit is silently N times what you configured. Scale from 3 pods to 10 and your limit tripled without anyone touching the config.
The counter has to live somewhere all instances can reach. Redis is the standard choice because it is fast and its operations are atomic. That last word is doing a lot of work. Consider the tempting but broken version:
// BROKEN: two round trips with a gap in the middle
const n = Number(await redis.get(key)); // instance A and B both read 99
if (n < limit) {
await redis.incr(key); // both increment → 101, both allowed
}Under load, two requests read 99 at the same instant, both decide they are under the limit, and both proceed. That is a time-of-check-to-time-of-use race, and it shows up precisely when you are busiest, which is when the limiter matters most. The fix is to make read-decide-write a single indivisible operation.
For the plainest fixed window, one INCR does it, because INCR returns the post-increment value atomically:
local n = redis.call('INCR', KEYS[1])
if n == 1 then redis.call('EXPIRE', KEYS[1], ARGV[1]) end
return n -- caller allows when n <= limitFor anything with branching logic (the token bucket’s refill, the sliding log’s prune-count-add), wrap the whole sequence in a Lua script and run it with EVAL. Redis executes a script to completion with nothing interleaved, so your read, your decision, and your write happen as one step, in one round trip. This is why every serious Redis limiter is a small Lua script and not a handful of separate commands.
The response: say no in a way clients can obey
A good limiter does not just drop the request. It tells the caller what happened and when to come back, so a well-behaved client backs off instead of hammering. The status code for this is 429 Too Many Requests, and it should carry a couple of headers.
Retry-After is the one that matters most. It tells the client how long to wait, either as a number of seconds or an HTTP date. A client’s retry logic can read it and sleep exactly that long instead of guessing.
HTTP/1.1 429 Too Many Requests
Retry-After: 5
RateLimit: "default";r=0;t=5
RateLimit-Policy: "default";q=100;w=60
Content-Type: application/problem+json
{
"type": "https://iana.org/assignments/http-problem-types#quota-exceeded",
"title": "Too Many Requests",
"status": 429,
"detail": "Rate limit exceeded. Retry in 5 seconds."
}The RateLimit and RateLimit-Policy fields let a good client see its budget on every response, not only after it gets blocked, so it can pace itself and never hit the wall. In the current IETF draft, RateLimit-Policy: "default";q=100;w=60 advertises the policy (a quota q of 100 over a window w of 60 seconds), and RateLimit: "default";r=0;t=5 reports the live state (r, remaining quota, is 0; t, seconds until reset, is 5).
Defence in layers
One limiter is rarely the whole answer. The strongest setups stack cheap-and-broad in front of precise-and-narrow, so each layer only handles what the layer before it let past.
The edge (CDN or WAF) is your first and cheapest line. A rule there rejects a flood at a data center near the attacker, before it ever reaches your origin, so a volumetric attack costs you nothing. On edge platforms like Cloudflare, a rate-limiting rule is a few fields and it is available even on free plans. It counts by a characteristic you choose and blocks over a threshold:
# Edge rate-limiting rule (illustrative)
when http.request.uri.path eq "/auth/forgot-password"
count by ip.src
rate 5 requests / 60 seconds
then block for 600 seconds # origin never sees the 6th requestThe edge is blunt on purpose: it is great at stopping obvious floods, and it cannot see your business logic. That is the next layer’s job.
The app limiter is where the precise, identity-aware limits live, because only your app knows the authenticated user, the plan tier, and which route is expensive. This is where the token bucket keyed on user:route sits. The two layers are not competitors; running both is the correct answer.
How this site does it
The password-reset story at the top is not hypothetical; it is the failure mode this project is built to avoid, since the buy-and-deliver flow sends real email and costs real money. The defence is three plain layers, and none of them is clever.
An edge rule caps requests per IP to the sensitive paths, so a crude flood dies at the CDN. An app-level limiter, a token bucket keyed on the authenticated user for logged-in traffic and on IP for anonymous, guards the specific handlers that do work. And a per-email cooldown sits on the reset and delivery endpoints themselves: one message per address per window, regardless of who asks or how many times.
// One reset email per address per 15 minutes, no matter who triggers it.
// SET NX only succeeds if the key does not already exist → atomic claim.
const key = `cooldown:reset:${sha256(email)}`;
const claimed = await redis.set(key, '1', { NX: true, EX: 900 });
if (claimed) {
await sendResetEmail(email); // first request in the window: actually send
}
// Respond the same either way, so nobody can probe which addresses exist.
return reply.code(200).send({ ok: true });Two details do the heavy lifting. The email is hashed before it becomes a key, so the rate-limit store never holds raw addresses. And the response is identical whether or not a send happened, because a reset endpoint that answers differently for real and fake accounts is an account-enumeration oracle. The cooldown stops the mailbomb; the constant response stops the leak. See security in depth for why that constant response matters.
Try it: a rate limiter lab
Pick an algorithm, set the limit and window, then fire requests and watch the timeline fill with green (allowed) and red (blocked) dots. The guided way to see the fixed-window bug: choose Fixed window, click Jump before boundary, fire a burst, click Cross boundary, fire another burst. Twice your limit sails through in a blink. Switch to Sliding counter and repeat: the second burst is mostly blocked.
Notice how the token bucket never shows the double-limit seam, because it has no seam. That is the whole reason it tends to be the right default for a public API.
Summary
- Every public endpoint is a free compute button for a stranger. Rate limiting is how you answer “you have done enough for now” cheaply, before the expensive work runs.
- What you key on matters more than the algorithm. IP is blunt (CGNAT, offices, botnets); user id or API key is precise but needs auth. Limit per endpoint, and give login, password reset, email/SMS and anything that calls an LLM or costs money their own tight limits.
- Fixed window is cheap but leaks double the limit across the boundary. Sliding window log is exact but stores a timestamp per request. Sliding window counter blends two counters (
prev × (1 − elapsed) + current) and is the practical default. Token bucket allows deliberate bursts up to capacity and caps the sustained rate at the refill rate, and is the usual right answer. Leaky bucket smooths output for a fragile downstream. - Counters must be shared. N instances with local state means N times your limit. Keep the count in Redis and make read-decide-write atomic with
INCRor a LuaEVAL, or a race lets extra requests through under load. - Answer with
429plusRetry-Afterso good clients back off. The IETFRateLimit/RateLimit-Policyheaders advertise the budget on every response, but they are still a draft;X-RateLimit-*is the older common form, so pick one and keep it stable. - Stack the layers. An edge rule sheds floods for free, an app limiter guards business logic, and both together is correct. Never put a limiter in front of your health checks or trusted internal traffic.