Right now.
Open any login page and the form looks defenseless. A username field, a password field, a button. Nothing about it looks guarded. But sitting in front of that form is a piece of infrastructure that makes a decision on every single request: is this a human who mistyped their password, or a script that just tried its 4,000th combination this hour?
That piece of infrastructure is a rate limiter, and it's one of the most quietly important things a backend engineer builds. Get it wrong and your login endpoint is a free password-guessing range, open 24/7. Get it right, and an attacker trying a thousand passwords a minute gets the exact same response after attempt 101, every time: blocked, with nothing to learn from.
Let's build one from scratch, watch the first version break in a very specific way, and fix it twice.
Notice what happened once you crossed 100: every attempt after that got the identical reply, instantly, without the server even looking at whether the password was right. That's the whole point. The rate limiter sits in front of the real authentication logic and never lets a flood reach it.
Before the algorithms — what the server actually tracks
A rate limiter needs exactly two numbers per user (or per IP, or both): how many requests have come in, and how much time has passed. Every algorithm below is just a different answer to one question — when exactly does the count go back to zero?
The Fixed Window Counter
The obvious first version: pick a window size, say 60 seconds. Give every user a counter. Every request increments it. If the counter exceeds the limit, reject with a 429. When the clock crosses into the next 60-second window, the counter resets to zero.
# Fixed window counter
WINDOW_SECONDS = 60
LIMIT = 100
def allow_request(user_id):
window = current_unix_time() // WINDOW_SECONDS
key = f"{user_id}:{window}"
count = increment(key)
if count == 1:
set_expiry(key, WINDOW_SECONDS)
return count <= LIMIT
Clean, cheap, and almost right. The flaw is in that word "window" — it's a hard clock boundary, not a rolling measure of the last 60 seconds. Watch what happens to an attacker who's timing it:
100 requests land in the last two seconds of Window A. The clock ticks over. The counter resets to zero. Another 100 land in the first two seconds of Window B. From the rate limiter's point of view, both windows stayed under their limit — it did its job perfectly, twice. From the actual world's point of view, 200 requests just got through in about four seconds, against a limit that was supposed to mean "100 per minute." The limiter wasn't wrong. The clock boundary was just a door that opens twice if you knock at the right second.
The Token Bucket
The fix is to stop resetting the counter all at once and instead let it drain and refill continuously. Picture a bucket that holds up to 100 tokens. Every request costs one token. If the bucket is empty, the request is rejected. A new token drips back into the bucket every 0.6 seconds — because 100 tokens over 60 seconds works out to one token every 0.6 seconds.
# Token bucket
class TokenBucket:
def __init__(self, capacity, refill_per_second):
self.capacity = capacity
self.tokens = capacity
self.refill_per_second = refill_per_second
self.last_check = now()
def allow(self):
elapsed = now() - self.last_check
self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_per_second)
self.last_check = now()
if self.tokens >= 1:
self.tokens -= 1
return True
return False
There's no clock boundary to exploit anymore, because there's no fixed boundary at all — the refill is continuous. A user can still burst (if the bucket is full, they can fire off 100 requests back to back, which is fine, that's a legitimate retry-fast scenario like someone fat-fingering their password three times in a row), but the sustained rate can never exceed one request per 0.6 seconds over any stretch of time, boundary or not.
This is close to what most real systems run: GitHub's API, Stripe's API, and most login endpoints you've ever hit a "too many attempts" wall on are all doing some flavor of token bucket. It's cheap to compute, needs only two numbers per user (token count and a timestamp), and has no boundary to exploit.
What happens with more than one server
Here's the part that doesn't show up until production. No real app runs on one server — there's a load balancer spreading requests across a fleet. If each server keeps its token bucket in its own local memory, the rate limiter is still "working" on every individual server. It just isn't limiting anything real.
Keep clicking past 100 and nothing stops you, because the load balancer is spreading your requests across three independent buckets. With 3 servers, your effective limit isn't 100 a minute — it's 300. With 50 servers behind an auto-scaling group, it's 5,000. The limiter isn't broken on any individual server. The limit itself only existed locally, and locally stopped meaning anything the moment there was more than one "local."
The fix — one shared counter, not three
The solution is to stop storing the bucket in each server's process memory and move it somewhere every server can see: a single shared, in-memory data store. Redis is the standard choice here — reads and writes complete in well under a millisecond, and because Redis executes each command atomically on a single thread, two servers hitting INCR on the same key in the same instant still can't both land on the same count. Redis serializes them, one after the other, no matter how many servers are asking at once.
# Shared rate limiter — every server runs this exact code
import redis
r = redis.Redis(host='ratelimit.internal', port=6379)
def allow_request(user_id):
key = f"ratelimit:{user_id}"
count = r.incr(key) # atomic increment, returns the new value
if count == 1:
r.expire(key, 60) # first request in this window starts the clock
return count <= 100
Same three servers, same random routing — but now the number that matters lives in exactly one place. It caps at 100 no matter which server happened to receive the request, which is the entire point: the limit is a property of the user, not of whichever machine answered the phone.
Notice the bottom branch. A wrong password still counts against the limit — it has to, otherwise the rate limiter would only ever stop correct guesses, which is exactly backwards. The limiter doesn't know or care whether a password was right. It only counts attempts.
Fixed Window vs Token Bucket vs the rest
Token bucket isn't the only fix for the boundary problem — it's just the most common one for login endpoints specifically. Two other approaches show up often enough to know by name:
| Algorithm | How it works | Weakness | Typical use |
|---|---|---|---|
| Fixed Window Counter | Hard reset every N seconds | 2x burst possible at the boundary | Internal APIs where exact limits don't matter much |
| Token Bucket | Bucket drains per request, refills continuously | Still allows a full-capacity burst if the bucket was sitting full | Login endpoints, Stripe's API, GitHub's API |
| Sliding Window Log | Stores a timestamp per request, counts how many fall in the trailing N seconds | Memory grows with request volume — one entry per request | Anywhere the exact limit genuinely matters |
| Leaky Bucket | Requests queue up and leak out at one fixed rate | Smooths traffic into a queue instead of just rejecting excess | Traffic shaping, not usually login forms |
Which one to pick
For a login form specifically: token bucket, backed by one shared Redis instance, keyed by both IP and account. Pair every rejection with a proper 429 Too Many Requests status and a Retry-After header, so a legitimate client — a mobile app with a retry loop, not an attacker — knows exactly how long to wait instead of hammering the endpoint harder. Log every rejection separately from every failed login; the shape of those two logs tells you whether you're looking at a user who forgot their password or a script working through a list.
The flashy version of this story is one attacker firing a thousand requests a second. The boring reality that actually stops them is one counter, sitting in Redis, answered correctly a thousand times a second, no matter which of your servers happens to be asking.
Srikanth Badavath