A single misbehaving client — buggy, misconfigured, or malicious — can consume a disproportionate share of a service’s capacity if nothing stops it. Rate limiting is the mechanism that enforces a cap before that happens.
What rate limiting actually protects against
A rate limiter restricts how many requests a given client (identified by API key, IP address, user account, or similar) can make within a defined time period, rejecting or delaying requests beyond that limit. It serves multiple purposes at once: protecting a service’s own capacity from being overwhelmed by a single caller, protecting downstream dependencies that same service calls from being overwhelmed in turn, and — connecting to Understanding DDoS Attacks and Mitigation — blunting application-layer attacks that send a high volume of individually legitimate-looking requests.
Fixed window: simple, but bursty at the boundary
A fixed window algorithm counts requests within fixed, non-overlapping time intervals — for example, resetting the count to zero at the start of every minute. It’s simple to implement and reason about, but has a specific, well-known weakness at window boundaries: a client could send its full limit of requests in the final moments of one window, and its full limit again in the first moments of the next, resulting in roughly double the intended rate within a short span straddling the boundary, even though each individual window’s count stayed within its own limit.
Sliding window: smoothing out the boundary problem
A sliding window algorithm addresses this by evaluating the limit over a continuously moving time window, rather than resetting sharply at fixed intervals — approximating the count of requests in, say, “the last 60 seconds” measured from right now, rather than “this calendar minute.” This smooths out the boundary-burst problem fixed windows have, at the cost of being somewhat more complex to implement and, depending on the specific approach, an approximation rather than an exact count.
Token bucket: allowing controlled bursts
A token bucket algorithm works differently: a bucket holds a maximum number of tokens, refilled at a steady rate over time, and each request consumes one token; a request is only allowed if a token is available. This naturally permits short bursts of traffic — up to the bucket’s full capacity at once — while still enforcing a steady average rate over time, since the bucket only refills gradually. Nginx’s documentation on rate limiting describes a version of this model in a real, widely deployed implementation. Token bucket is often preferred specifically because real traffic is rarely perfectly smooth, and a rate limiter that rejects a legitimate short burst just as readily as a sustained overload can be more disruptive to real users than necessary.
Choosing between them
Fixed window is the simplest to reason about and implement, and adequate when the boundary-burst issue isn’t a meaningful concern for the use case. Sliding window offers more accurate, smoother enforcement at moderate additional complexity. Token bucket is generally the best fit when legitimate traffic is naturally bursty and a limiter that’s overly strict about short bursts would reject real users unnecessarily — which is why it’s a common default choice for public-facing APIs.
Where rate limiting fits in a larger architecture
Rate limiting is commonly enforced at an API gateway, centralizing it the same way authentication and routing are centralized rather than reimplemented in every backend service. It’s also a complementary, not competing, defense alongside the circuit breakers and backoff discussed elsewhere on this site: rate limiting controls how much load a client is allowed to generate in the first place, while circuit breakers and backoff govern how a system responds once a dependency is already showing signs of being overwhelmed regardless.
Key takeaway
Rate limiting caps how much load any single client can place on a service, protecting both that service and whatever it depends on from being overwhelmed. Fixed window is simplest but bursts at window boundaries, sliding window smooths that out at some added complexity, and token bucket explicitly allows controlled bursts while still enforcing a steady average rate — making it the common choice where legitimate traffic is naturally uneven.
This article explains general rate limiting concepts; specific algorithms and defaults vary by implementation. See our disclaimer.