Hanad KubatVisit Website →
← Back to blog

API Rate Limiting: How to Design, Set and Enforce Limits

November 7, 2026
API Rate Limiting: How to Design, Set and Enforce Limits

API rate limiting is a rule that caps how many requests a client can send in a given period. Past the cap, the API rejects the request, usually with an HTTP 429 status. The short version of doing it well: pick one identity to count against (usually the customer account), start with a token bucket, put tighter limits on expensive endpoints, and tell clients exactly when they can retry.

I'm Hanad, a senior software engineer in Vienna. I build first real versions of SaaS products at a fixed price, and rate limiting is one of those things that is easy to skip in week one and painful to bolt on after launch. This guide covers how limits work, which algorithm to choose, how to set limits for different customers, and what to do when you're the one hitting a limit.

Hanad Kubat
Build Rate Limits Into Your First Version
Hanad builds working software products at a fixed price, so you can include API rate limiting in the product from the start.
Explore software development

Key Takeaways

  • A rate limit identifies the caller, checks recent usage against a policy, then allows or rejects the request. Rejected requests should get a 429 with a Retry-After header.
  • Token bucket allows short bursts and suits most customer-facing APIs. Leaky bucket smooths traffic and suits queues that protect a fragile downstream system.
  • In a SaaS product, key limits by tenant or API key, not by IP, and give expensive endpoints their own, lower limits.
  • With more than one server, counters must live in shared state such as Redis, or the limit will not be consistent.
  • On the client side, honor Retry-After, use exponential backoff with jitter, cap retries, and use idempotency keys for writes.

Table of Contents

What API rate limiting is and why APIs use it

Rate limits are restrictions on how many times a user or client can access a service within a period. OpenAI's documentation describes them with metrics like requests per minute, requests per day, tokens per minute and tokens per day. The same page lists the reasons: protection against abuse or misuse, fair access for everyone, and management of aggregate load on the infrastructure.

Those reasons are really one idea. Microsoft's Dev Proxy documentation frames limiting as a way to prevent overload, keep access fair, reduce abuse, control infrastructure cost and preserve service quality. Protecting the service and stopping one caller from monopolizing it are the same job.

For a SaaS founder there is a fifth reason that matters just as much: money. A traffic spike, accidental or malicious, can cause slowdowns, crashes or unexpectedly high infrastructure costs. If your API calls an LLM or a paid third-party service on every request, an unthrottled customer script can turn into your own invoice.

Limits also shape your pricing. Cloudflare's learning page notes that rate limiting can push developers to pay more, since they can often make only so many calls before upgrading. A free tier, a standard tier and an enterprise tier usually differ in exactly this.

What API rate limiting is and why APIs use it

How rate limiting works step by step

Every implementation, from a ten-line middleware to a dedicated gateway, does the same four things:

  1. Identify the caller. This is the key you count against: an API key, a user ID, a tenant ID, or an IP address.
  2. Look up the policy. What is this caller allowed? The answer may depend on their plan and on which endpoint they're hitting.
  3. Check recent usage. Compare what the caller has used against the allowance, using one of the algorithms below.
  4. Allow or reject. Accepted requests proceed. Excess requests are, per Postman's explanation, typically rejected with HTTP 429 Too Many Requests.

What the 429 response should contain

A bare 429 tells the client it did something wrong but not what to do. A useful one also carries headers. Postman lists X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset and Retry-After, so the client knows its allowance, what is left, and when to try again. GitHub's REST API, for example, answers with 429 and sends the three X-RateLimit headers with rate-limited responses. The n8n guide also points to Retry-After as the way to say how long to wait.

An example response:

HTTP/1.1 429 Too Many Requests
Retry-After: 12
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1767225612
Content-Type: application/json

{"error":"rate_limited","message":"Retry in 12 seconds."}

A machine-readable error body costs nothing and saves your customers' developers a support email.

Rate limiting algorithms: token bucket vs leaky bucket and the rest

The algorithm decides how a limit feels to the client. Same number on paper, very different behavior in practice.

AlgorithmHow it worksBurstsState to storeBest fit
Fixed windowCounter per time slot, reset at the boundaryCan double at window edgesOne counter plus a TTLSimple internal limits, login attempts
Sliding windowCounts requests over a rolling periodSmooth, no boundary spikeTimestamps or weighted countersFair customer-facing limits
Token bucketBucket refills at a steady rate, each request spends a tokenAllowed up to bucket sizeToken count plus last-refill timePublic and customer APIs
Leaky bucketRequests queue and drain at a constant rateAbsorbed or droppedA queue plus leak timingProtecting a fragile downstream system

The state column comes from the task-queues comparison, which describes fixed-window state as a counter plus TTL, token bucket as a token count plus last-refill timestamp, and leaky bucket as a queue plus leak timing.

Token bucket

Tokens accumulate over time up to a maximum capacity, and each request consumes one. When the bucket is empty, requests are denied until it refills, which allows controlled bursts while enforcing a steady average rate. Requestly calls it the model most commonly associated with public APIs because it tolerates short bursts but still holds a long-run average.

That matches real usage. A customer's dashboard loads and fires eight requests at once, then goes quiet. A strict per-second cap would punish normal behavior. A bucket of 20 refilling at 5 per second would not.

Leaky bucket

The metaphor is a bucket with a hole in the bottom. Traffic enters, the bucket drains at a fixed rate, and if requests arrive faster than that, the bucket fills and overflow is dropped. Output is smooth and constant, which Requestly notes is closer to throttling than to a hard reject cap. Kong's description makes the trade-off plain: token bucket preserves bursts, leaky bucket gives them up in exchange for smooth output.

Token bucket and leaky bucket behavior compared

Which one to choose

The Redisson glossary gives the most practical rule I've seen. If a human is waiting on the response, reject fast with a token bucket and let the client retry. If the work is asynchronous and the downstream system is what needs protecting, queue it with a leaky bucket.

In practice:

  • Customer-facing REST API: token bucket, or a sliding window if you want the simplest mental model to explain in docs.
  • Background jobs calling a third-party API with a hard cap: leaky bucket, usually as a queue worker with a fixed drain rate.
  • Login and password reset: a fixed or sliding window with a low count, kept separate from your general API limit.

Kong recommends sliding-window limiting when flexibility, performance and avoiding fixed-window boundary bursts matter. If you can't decide, a sliding window or token bucket will serve a young product well.

Setting rate limits for different customers in a SaaS product

This is where most tutorials stop and where real products get interesting. Searching for "api rate limits for customers" usually means one question: how do I give different people different allowances without writing a mess?

Pick the right key

A global bucket is the first mistake. One customer's burst then consumes everyone's allowance. STOA's SaaS guide recommends per-tenant, per-tier and per-endpoint policies rather than one shared bucket.

A sensible identity hierarchy: key authenticated traffic by tenant, account, API key or user, and use IP-based controls mainly for unauthenticated routes. One SaaS rate-limiting guide recommends tenant keys for authenticated B2B traffic and IP keys where no tenant is known. IP alone is a poor key for business customers, because a whole office can sit behind one address.

Store limits as data, not constants

Hard-coding "100 per minute" works until your first enterprise customer asks for 1,000. A Rails example from Tejas Bubane derives the account from the API token and looks up that account's configured limit instead. Do the same in any stack: a plan or rate_limit_policy field on the account, read at request time and cached briefly.

Limit expensive endpoints separately

Not all requests cost the same. Terminal49 documents a rolling limit of 100 requests per minute per API key or account, plus a separate 10-per-minute bucket for a refresh-container endpoint. Search, exports, report generation, file uploads and anything that calls an LLM deserve their own, lower limits.

Separate throughput limits from quotas

Zuplo distinguishes short-term rate limits from longer-term quotas: requests per second or minute on one side, daily, monthly or yearly consumption on the other. OpenAI's own model uses several limits at once, requests and tokens, per minute and per day. A single requests-per-minute number is rarely enough. A common tier design looks like this:

TierThroughputExpensive endpointsMonthly quota
FreeLow, small burstVery low or disabledSmall, hard stop
StandardMediumLower than general limitSized to plan
EnterpriseNegotiated, per contractNegotiatedNegotiated or pooled

The actual numbers should come from your own cost per request and your own traffic, not from another company's docs. Measure a week of real usage, then set the limit comfortably above your heaviest legitimate customer and below the point where your infrastructure or your margin suffers.

For AI features specifically, I've written about how OpenAI rate limits work and how to raise them, which is the other side of this problem: your product is the client being limited.

Best practices for production rate limiting

The STOA guide boils the list down to per-tenant and endpoint-specific limits, remaining and reset headers, a 429 with Retry-After, plus monitoring and testing before launch. A few details I'd add from building these.

Warn before you block

STOA suggests warning before hard throttling: expose remaining capacity, give Retry-After guidance, and return 429 only at the hard limit. Its example uses an 80% warning threshold. A customer who gets an email or a dashboard banner at 80% of quota upgrades. A customer who gets a surprise outage churns.

Use shared state when you have more than one server

Per-process counters are inconsistent when requests can reach different servers. Lorbic's write-up explains that a consistent distributed limit needs shared state such as Redis and atomic operations. A check-then-increment done in two steps will let bursts through under concurrency. The increment and the decision should happen in one atomic operation.

Decide what happens when the limiter fails

What if Redis is down? Lorbic frames this as a route-dependent choice: availability-oriented business limits may fail open, while security-sensitive controls such as login brute-force protection may fail closed. Write the decision down per route. Don't let it be an accident of whichever exception handler catches the error.

Make limits visible in your docs and your metrics

Document every limit, every header and the retry behavior you expect. Then count 429s per customer and per endpoint. A rising 429 rate for one customer is either a sales conversation or a bug in their integration, and you want to know which before they email you.

Don't forget the rest of the API surface

Limits are easier to explain when the API is consistent. If you're still shaping endpoints, what to decide before you build an MVP API covers the choices that make later limits simple. And if limits are one of several things breaking under real users, app stabilization strategies shows the order I'd fix things in.

A worked example: 10 requests per minute

A common question is how to limit an API to 10 requests per minute. The design choice is the algorithm, not the number.

  • Fixed window: a counter keyed by customer_id:minute, expiring after 60 seconds. Simple, but a client can send 10 at 12:00:59 and 10 more at 12:01:00.
  • Sliding window: count requests in the last 60 seconds. No boundary spike, slightly more state.
  • Token bucket: capacity 10, refill one token every 6 seconds. Allows a burst of 10, then steady one every 6 seconds.

The pseudocode for the token bucket is short:

// state per key: tokens, updatedAt
const now = Date.now();
const elapsed = (now - state.updatedAt) / 1000;
state.tokens = Math.min(10, state.tokens + elapsed / 6);
state.updatedAt = now;
if (state.tokens >= 1) { state.tokens -= 1; allow(); }
else { reject(429, { retryAfter: Math.ceil((1 - state.tokens) * 6) }); }

In production, run that read-modify-write atomically in your shared store, as described above, and compute Retry-After from the actual refill time so the header is honest.

What to do when you hit an API rate limit

The other half of the keyword is the reader on the receiving end of a 429. The fix is client discipline, not a bigger loop.

  1. Honor Retry-After. Terminal49 recommends waiting the stated time and avoiding immediate retries that burn more of your allowance.
  2. Back off exponentially with jitter when there is no Retry-After. Socure suggests delays starting around 1 to 2 seconds and doubling, with random jitter added so many clients don't retry in lockstep.
  3. Cap retries. Socure stops after about five attempts or one minute. Then surface the failure instead of looping.
  4. Use idempotency keys on writes, so a retried payment or record creation can't happen twice.
  5. Reduce demand. Batch calls, cache responses, and queue non-urgent work through a worker with a fixed drain rate.

If you're consuming an LLM API and your product needs more headroom, the practical steps are in my guide to checking, handling and raising OpenAI limits. Latency and throughput also interact: fixing scheduling before model tuning often frees more capacity than a higher tier.

Perspective: build limits in from the first version

I build first versions of SaaS products at a fixed price, and I add rate limiting early for a simple reason: it is cheap on day one and expensive on day 200. By then customers depend on your behavior, your pricing assumes certain usage, and every change is a negotiation.

The pattern I use is deliberately plain. One identity per request, resolved from the API key. A policy stored on the account. A token bucket in shared state. A 429 with Retry-After and a clear JSON body. Counters and 429s visible in a dashboard. Nothing exotic, because the next developer should read it in ten minutes and change a number without fear. That is also why I stick to widely known tools such as TypeScript, Node, SQL and Cloudflare Workers.

I also tell founders honestly when they don't need to build this at all. If your API sits behind a managed gateway that already offers limits, use that and spend the time on your product. When the limits need to know about your plans, tenants and costs, that logic belongs in code you own from the first commit.

Rate limiting is also where rescued prototypes often fall over: no per-customer keys, no retry guidance, one noisy user taking everything down. Rebuilding that properly is a normal part of the work I do.

Hanad Kubat

Need rate limiting in a real product?

If your prototype has no limits, or limits that fall over under real users, I can build them as part of a proper first version or a rebuild. I work as a single senior engineer on fixed scope in two to four weeks, you see working progress before paying, and you own the code from the first commit. Builds start from €12,000, and the price is possible because there is no agency overhead, no project-manager layer and no offshore markup. See what I build and how it's priced.

For identity and permissions, which pair naturally with per-tenant limits, see role-based access control for SaaS.

FAQ

What is a good rate limit for an API?

There is no universal number. Start from measured usage: set the limit comfortably above your heaviest legitimate customer and below the point where cost or stability suffers. Then give expensive endpoints lower limits and larger customers negotiated ones, as the tiered approach above shows.

Should I use token bucket or leaky bucket?

Use a token bucket when a person or client is waiting on the response and short bursts are normal. Use a leaky bucket when you're feeding asynchronous work into a downstream system that can't tolerate spikes. Most customer-facing APIs are the first case.

What status code should a rate-limited request return?

Return HTTP 429 Too Many Requests, ideally with Retry-After and the X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset headers, so clients know when to retry. A short JSON error body makes it easier to handle in code.

How do I fix an "API rate limit reached" error?

Wait for the time in Retry-After, or back off exponentially with jitter if it's missing. Cap your retries and use idempotency keys on writes. Longer term, batch and cache requests, or move to a plan with a higher limit.

Should rate limits be per IP or per customer?

For authenticated B2B traffic, key by tenant, account or API key, because many users can share one IP address. Keep IP-based limits for unauthenticated routes such as sign-up and login.

Sources