Design an API rate limiter

Protect an API from overload and abuse: compare fixed windows, sliding logs, sliding window counters and token buckets, make limits work across many servers with Redis, then build HTTP middleware in Go with a live demo page.

1. Requirements#

Functional: limit requests per client (API key, user ID or IP) by rules such as "100 per minute"; reject excess requests with 429 Too Many Requests; tell clients their remaining quota.

Non-functional: the check adds under a millisecond; the limit is enforced across all servers; a limiter failure does not take the API down; memory per client is small (millions of clients).

2. The algorithms#

Algorithm	State per client	Accuracy	Bursts	Notes
fixed window counter	1 counter + window start	edges allow 2×	at window edges	simplest; `INCR` + `EXPIRE` in Redis
sliding window log	every timestamp in the window	exact	none beyond limit	memory grows with the limit
sliding window counter	2 counters	close approximation	smoothed	weights the previous window's count by overlap
token bucket	tokens + last refill time	exact for rate + burst	up to capacity	the most common choice (Stripe, AWS, `x/time/rate`)
leaky bucket	a queue	smooth output	absorbed by the queue	shapes traffic instead of rejecting it

Token bucket: a bucket holds up to capacity tokens and gains rate tokens per second. A request takes one token or is rejected. You never need a timer: when a request arrives, add elapsed × rate tokens (capped), then try to take one.

InteractiveToken bucket rate limiterThe bucket refills at a steady rate up to its capacity. Each request spends one token; with no tokens, it is rejected (HTTP 429).

tokens 3

requests

0.5s

1.2s

2.5s

2.6s

2.7s

The bucket starts full with 3 tokens, refilling 1/s.

1 / 13

Sliding window counter: keep the count for the current fixed window and the previous one. Estimate the requests in the last 60 seconds as current + previous × (fraction of the previous window still inside the sliding window). Two integers per client, and the edge-burst problem mostly disappears.

algorithms.go

package main
 
import "fmt"
 
// Each limiter answers Allow(t) for a request at time t (in seconds).
 
type FixedWindow struct {
	limit, window int
	start, count  int
}
 
func (f *FixedWindow) Allow(t int) bool {
	if w := t / f.window * f.window; w != f.start {
		f.start, f.count = w, 0 // a new window: reset
	}
	if f.count < f.limit {
		f.count++
		return true
	}
	return false
}
 
type SlidingCounter struct {
	limit, window       int
	curStart, cur, prev int
}
 
func (s *SlidingCounter) Allow(t int) bool {
	w := t / s.window * s.window
	switch {
	case w == s.curStart+s.window:
		s.prev, s.cur, s.curStart = s.cur, 0, w
	case w != s.curStart:
		s.prev, s.cur, s.curStart = 0, 0, w
	}
	overlap := float64(s.window-(t-w)) / float64(s.window) // share of the previous window still in range
	if float64(s.prev)*overlap+float64(s.cur) < float64(s.limit) {
		s.cur++
		return true
	}
	return false
}
 
func main() {
	// Limit: 5 requests per 10 seconds. A client sends 5 at t=9 and 5 at t=10.
	burst := []int{9, 9, 9, 9, 9, 10, 10, 10, 10, 10}
	fw := &FixedWindow{limit: 5, window: 10}
	sc := &SlidingCounter{limit: 5, window: 10}
	allowedFW, allowedSC := 0, 0
	for _, t := range burst {
		if fw.Allow(t) {
			allowedFW++
		}
		if sc.Allow(t) {
			allowedSC++
		}
	}
	fmt.Println("fixed window allowed", allowedFW, "of 10 within 2 seconds (limit 5 per 10s)")
	fmt.Println("sliding counter allowed", allowedSC)
}

Output

fixed window allowed 10 of 10 within 2 seconds (limit 5 per 10s)
sliding counter allowed 5

3. Where the limiter runs#

Drawing diagram…

At the edge (API gateway, CDN, load balancer): cheap protection against floods before they reach your code.
In the application (middleware): limits that need business context, such as the user's plan or the cost of an endpoint.
Shared state: with many app servers, a per-server limit lets a client get N × the limit. Store counters in Redis, keyed by client and window, with expirations so idle clients cost nothing.

4. Telling clients what happened#

Return 429 Too Many Requests with Retry-After: <seconds>, and on every response the remaining quota. The IETF RateLimit headers (RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset) are becoming the standard; many APIs still use X-RateLimit-*. Well-behaved clients read them and back off.

5. Deep dives#

Atomicity across servers: reading the counter and writing it back from two servers at once lets both through. Run the whole check in Redis as one Lua script (step 6), which Redis executes atomically.
Failure mode: if Redis is unreachable, fail open with a conservative in-memory limit per server rather than rejecting everything.
Hot keys: a single huge customer can overload one Redis shard. Give them a dedicated key range, or approximate with local token buckets that sync periodically.
Cost-based limits: an expensive search can cost 10 tokens and a cheap read 1.

Build it: rate-limiting middleware in Go#

You will build a reusable limiter package with a fake clock for exact tests, an HTTP middleware that sets the standard headers, and a demo page that fires requests and shows which were allowed.

ratelimit/
├── go.mod
├── main.go                       demo API: /api/ping is limited per client
├── main_test.go
├── internal/limiter/bucket.go    token bucket + keyed limiter with cleanup
├── internal/limiter/bucket_test.go
└── web/index.html                fire requests, watch allowed vs rejected

Step 1: module

Shell

mkdir ratelimit && cd ratelimit && go mod init ratelimit
mkdir -p internal/limiter web

go.mod

module ratelimit
 
go 1.24

Step 2: the token bucket, with an injectable clock

Time makes tests flaky. The limiter takes a now func() time.Time, so tests control time exactly instead of sleeping.

internal/limiter/bucket.go

// Package limiter implements per-key token bucket rate limiting.
package limiter
 
import (
	"math"
	"sync"
	"time"
)
 
// Decision is the result of one Allow call.
type Decision struct {
	Allowed    bool
	Remaining  int           // whole tokens left after this request
	RetryAfter time.Duration // how long until one token is available (when rejected)
}
 
type bucket struct {
	tokens float64
	last   time.Time
}
 
// Keyed holds one token bucket per key (user, API key or IP).
type Keyed struct {
	mu       sync.Mutex
	capacity float64
	rate     float64 // tokens per second
	buckets  map[string]*bucket
	now      func() time.Time
}
 
// New creates a limiter: bursts up to capacity, sustained rate per second.
func New(capacity int, ratePerSecond float64, now func() time.Time) *Keyed {
	if now == nil {
		now = time.Now
	}
	return &Keyed{capacity: float64(capacity), rate: ratePerSecond, buckets: map[string]*bucket{}, now: now}
}
 
// Allow spends cost tokens for key if available.
func (k *Keyed) Allow(key string, cost int) Decision {
	k.mu.Lock()
	defer k.mu.Unlock()
	now := k.now()
	b, ok := k.buckets[key]
	if !ok {
		b = &bucket{tokens: k.capacity, last: now} // new clients start full
		k.buckets[key] = b
	}
	elapsed := now.Sub(b.last).Seconds()
	b.tokens = math.Min(k.capacity, b.tokens+elapsed*k.rate)
	b.last = now
	c := float64(cost)
	if b.tokens >= c {
		b.tokens -= c
		return Decision{Allowed: true, Remaining: int(b.tokens)}
	}
	wait := (c - b.tokens) / k.rate
	return Decision{Remaining: int(b.tokens), RetryAfter: time.Duration(math.Ceil(wait*1000)) * time.Millisecond}
}
 
// Cleanup drops buckets that have been full (idle) for at least idle, so memory stays bounded.
func (k *Keyed) Cleanup(idle time.Duration) int {
	k.mu.Lock()
	defer k.mu.Unlock()
	now := k.now()
	removed := 0
	for key, b := range k.buckets {
		full := b.tokens+now.Sub(b.last).Seconds()*k.rate >= k.capacity
		if full && now.Sub(b.last) >= idle {
			delete(k.buckets, key)
			removed++
		}
	}
	return removed
}
 
// Len reports how many keys are tracked.
func (k *Keyed) Len() int {
	k.mu.Lock()
	defer k.mu.Unlock()
	return len(k.buckets)
}

internal/limiter/bucket_test.go

package limiter
 
import (
	"testing"
	"time"
)
 
type fakeClock struct{ t time.Time }
 
func (c *fakeClock) Now() time.Time          { return c.t }
func (c *fakeClock) Advance(d time.Duration) { c.t = c.t.Add(d) }
 
func TestBurstThenRate(t *testing.T) {
	clk := &fakeClock{t: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)}
	l := New(3, 1, clk.Now) // burst 3, then 1 per second
 
	for i := range 3 {
		if d := l.Allow("alice", 1); !d.Allowed {
			t.Fatalf("request %d of the burst rejected", i)
		}
	}
	d := l.Allow("alice", 1)
	if d.Allowed || d.RetryAfter != time.Second {
		t.Fatalf("4th request: %+v, want rejected with RetryAfter 1s", d)
	}
	if !l.Allow("bob", 1).Allowed {
		t.Fatal("bob has his own bucket")
	}
 
	clk.Advance(500 * time.Millisecond)
	if l.Allow("alice", 1).Allowed {
		t.Fatal("half a token is not enough")
	}
	clk.Advance(500 * time.Millisecond)
	if !l.Allow("alice", 1).Allowed {
		t.Fatal("one second later a token is back")
	}
}
 
func TestCostAndCap(t *testing.T) {
	clk := &fakeClock{t: time.Unix(0, 0)}
	l := New(10, 2, clk.Now)
	if !l.Allow("k", 10).Allowed || l.Allow("k", 1).Allowed {
		t.Fatal("cost 10 should drain the bucket")
	}
	clk.Advance(time.Hour) // refill is capped at capacity, not 7200 tokens
	if d := l.Allow("k", 1); d.Remaining != 9 {
		t.Fatalf("remaining = %d, want 9", d.Remaining)
	}
}
 
func TestCleanup(t *testing.T) {
	clk := &fakeClock{t: time.Unix(0, 0)}
	l := New(5, 1, clk.Now)
	l.Allow("a", 5)
	l.Allow("b", 1)
	clk.Advance(10 * time.Second)
	if n := l.Cleanup(5 * time.Second); n != 2 || l.Len() != 0 {
		t.Fatalf("cleanup removed %d, %d left", n, l.Len())
	}
}

Step 3: the middleware and demo API

Middleware wraps any handler. It identifies the client (an X-API-Key header if present, otherwise the IP), asks the limiter, sets the headers, and either calls the next handler or returns 429. A background goroutine runs Cleanup every minute.

main.go

package main
 
import (
	_ "embed"
	"encoding/json"
	"log/slog"
	"net"
	"net/http"
	"os"
	"strconv"
	"time"
 
	"ratelimit/internal/limiter"
)
 
//go:embed web/index.html
var indexHTML []byte
 
// clientKey prefers an API key; otherwise it uses the client IP.
// Behind a trusted proxy, read X-Forwarded-For instead (and only then).
func clientKey(r *http.Request) string {
	if k := r.Header.Get("X-API-Key"); k != "" {
		return "key:" + k
	}
	host, _, err := net.SplitHostPort(r.RemoteAddr)
	if err != nil {
		host = r.RemoteAddr
	}
	return "ip:" + host
}
 
func rateLimit(l *limiter.Keyed, limit int, next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		d := l.Allow(clientKey(r), 1)
		h := w.Header()
		h.Set("RateLimit-Limit", strconv.Itoa(limit))
		h.Set("RateLimit-Remaining", strconv.Itoa(d.Remaining))
		if !d.Allowed {
			secs := int((d.RetryAfter + time.Second - 1) / time.Second) // round up
			h.Set("Retry-After", strconv.Itoa(secs))
			h.Set("Content-Type", "application/json")
			w.WriteHeader(http.StatusTooManyRequests)
			json.NewEncoder(w).Encode(map[string]any{"error": "rate limit exceeded", "retry_after_seconds": secs})
			return
		}
		next.ServeHTTP(w, r)
	})
}
 
func newServer(l *limiter.Keyed, limit int) http.Handler {
	mux := http.NewServeMux()
	mux.HandleFunc("GET /{$}", func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "text/html; charset=utf-8")
		w.Write(indexHTML)
	})
	ping := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "application/json")
		json.NewEncoder(w).Encode(map[string]string{"pong": time.Now().UTC().Format(time.RFC3339Nano)})
	})
	mux.Handle("GET /api/ping", rateLimit(l, limit, ping))
	return mux
}
 
func main() {
	const burst, perSecond = 5, 1.0
	l := limiter.New(burst, perSecond, nil)
	go func() {
		for range time.Tick(time.Minute) {
			if n := l.Cleanup(5 * time.Minute); n > 0 {
				slog.Info("cleanup", "removed", n, "tracked", l.Len())
			}
		}
	}()
	port := os.Getenv("PORT")
	if port == "" {
		port = "8080"
	}
	slog.Info("listening", "port", port, "burst", burst, "per_second", perSecond)
	srv := &http.Server{Addr: ":" + port, Handler: newServer(l, burst), ReadHeaderTimeout: 5 * time.Second}
	if err := srv.ListenAndServe(); err != nil {
		slog.Error("server", "err", err)
		os.Exit(1)
	}
}

main_test.go

package main
 
import (
	"net/http"
	"net/http/httptest"
	"testing"
	"time"
 
	"ratelimit/internal/limiter"
)
 
func TestMiddleware(t *testing.T) {
	now := time.Unix(0, 0)
	l := limiter.New(2, 1, func() time.Time { return now })
	h := newServer(l, 2)
 
	get := func(key string) *httptest.ResponseRecorder {
		req := httptest.NewRequest("GET", "/api/ping", nil)
		req.Header.Set("X-API-Key", key)
		rec := httptest.NewRecorder()
		h.ServeHTTP(rec, req)
		return rec
	}
	for i := range 2 {
		if rec := get("team-a"); rec.Code != 200 {
			t.Fatalf("request %d: %d", i, rec.Code)
		}
	}
	rec := get("team-a")
	if rec.Code != http.StatusTooManyRequests || rec.Header().Get("Retry-After") != "1" {
		t.Fatalf("3rd: %d, Retry-After %q", rec.Code, rec.Header().Get("Retry-After"))
	}
	if rec.Header().Get("RateLimit-Remaining") != "0" {
		t.Errorf("remaining header = %q", rec.Header().Get("RateLimit-Remaining"))
	}
	if get("team-b").Code != 200 {
		t.Error("another key must not be affected")
	}
	now = now.Add(time.Second)
	if get("team-a").Code != 200 {
		t.Error("after one second, team-a should get a token back")
	}
}
 
func TestClientKeyFallsBackToIP(t *testing.T) {
	req := httptest.NewRequest("GET", "/", nil)
	req.RemoteAddr = "203.0.113.7:51234"
	if got := clientKey(req); got != "ip:203.0.113.7" {
		t.Errorf("clientKey = %q", got)
	}
}

Step 4: a page to watch it work

The page fires a burst of requests and draws a square per request, green for allowed and red for rejected, with the headers the server returned.

web/index.html

HTML

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Rate limiter demo</title>
  <style>
    body { font-family: system-ui, sans-serif; max-width: 680px; margin: 3rem auto; padding: 0 1rem; }
    button { padding: .5rem .9rem; margin-right: .4rem; border-radius: 8px; border: 1px solid #999; background: #fff; cursor: pointer; }
    #grid { display: flex; flex-wrap: wrap; gap: 6px; margin: 1rem 0; }
    .sq { width: 28px; height: 28px; border-radius: 6px; display: grid; place-items: center; font: 12px monospace; color: #fff; }
    .ok { background: #79740e; } .no { background: #9d0006; }
    #log { font: 13px/1.5 monospace; white-space: pre-wrap; background: #f4f4f4; padding: .8rem; border-radius: 8px; }
  </style>
</head>
<body>
  <h1>Token bucket: burst 5, refill 1/s</h1>
  <p>
    <button id="one">Send 1</button>
    <button id="ten">Send 10 at once</button>
    <button id="steady">Send 1 per second for 8s</button>
  </p>
  <div id="grid"></div>
  <div id="log"></div>
  <script>
    let n = 0;
    async function send() {
      const res = await fetch("/api/ping");
      n++;
      const sq = document.createElement("div");
      sq.className = "sq " + (res.ok ? "ok" : "no");
      sq.textContent = n;
      document.getElementById("grid").append(sq);
      const line = `#${n} ${res.status} remaining=${res.headers.get("RateLimit-Remaining")}` +
        (res.ok ? "" : ` retry-after=${res.headers.get("Retry-After")}s`);
      const log = document.getElementById("log");
      log.textContent = line + "\n" + log.textContent;
    }
    document.getElementById("one").onclick = send;
    document.getElementById("ten").onclick = () => { for (let i = 0; i < 10; i++) send(); };
    document.getElementById("steady").onclick = () => {
      let k = 0;
      const t = setInterval(() => { send(); if (++k === 8) clearInterval(t); }, 1000);
    };
  </script>
</body>
</html>

Step 5: run it

Shell

go test -race ./...
go run .
# open http://localhost:8080 and press "Send 10 at once": 5 green, 5 red
for i in $(seq 1 7); do curl -s -o /dev/null -w "%{http_code} " localhost:8080/api/ping; done; echo
# 200 200 200 200 200 429 429

Step 6: sharing limits across servers with Redis

With several copies of the server, move the bucket into Redis. This Lua script does the same arithmetic as Allow, atomically, and sets an expiry so idle keys disappear on their own:

Lua

-- KEYS[1] = bucket key; ARGV = capacity, rate per second, now (ms), cost
local cap, rate, now, cost = tonumber(ARGV[1]), tonumber(ARGV[2]), tonumber(ARGV[3]), tonumber(ARGV[4])
local b = redis.call("HMGET", KEYS[1], "tokens", "ts")
local tokens = tonumber(b[1]) or cap
local ts = tonumber(b[2]) or now
tokens = math.min(cap, tokens + (now - ts) / 1000 * rate)
local allowed = 0
if tokens >= cost then tokens = tokens - cost; allowed = 1 end
redis.call("HSET", KEYS[1], "tokens", tokens, "ts", now)
redis.call("PEXPIRE", KEYS[1], math.ceil(cap / rate * 1000))
return { allowed, math.floor(tokens) }

In Go, load it once with go-redis (redis.NewScript(src)) and call script.Run(ctx, rdb, []string{"rl:" + key}, capacity, rate, nowMillis, cost). Wrap the call with a short timeout; on error, fall back to the in-memory limiter (fail open with local protection).

Summary#

Fixed windows are simple but allow double bursts at edges; sliding counters and token buckets fix that.
A token bucket's capacity sets the burst and its refill rate sets the average; it needs only two numbers per client.
Limit at the edge for floods and in the app for business rules; share state in Redis with atomic scripts.
Return 429 with Retry-After and quota headers; fail open with a local fallback when the shared store is down.
The build injects a clock so every limiter behavior is tested exactly, with no sleeps.