resilience • Intermediate

Distributed Rate Limiting: Token Bucket vs Sliding Window Counter

Rate limiters protect downstream services from cascading failure and abusive traffic. Sliding Window Counters combine boundary precision with O(1) memory overhead by interpolating previous window weight.

Intuitive Mental Model

The Nightclub Bouncer: The Token Bucket bouncer adds 5 wristbands into a bowl every second. Guests take 1 wristband to enter. If a group of 10 arrives instantly, they consume the accumulated burst without waiting, as long as tokens exist.

Architecture Blueprint & CodeProduction Standard

// Redis Atomic Lua Script for Token Bucket:
const tokenBucketLua = `
  local key = KEYS[1]
  local maxTokens = tonumber(ARGV[1])
  local refillRate = tonumber(ARGV[2])
  local now = tonumber(ARGV[3])
  
  local data = redis.call('HMGET', key, 'tokens', 'lastRefill')
  local tokens = tonumber(data[1]) or maxTokens
  local lastRefill = tonumber(data[2]) or now
  
  local delta = math.max(0, now - lastRefill)
  tokens = math.min(maxTokens, tokens + delta * refillRate)
  
  if tokens >= 1 then
    tokens = tokens - 1
    redis.call('HMSET', key, 'tokens', tokens, 'lastRefill', now)
    return 1 -- Allowed
  else
    return 0 -- Rate limited (HTTP 429)
  end
`;

Key Architectural Takeaways

•Token Bucket: Ideal for handling bursty traffic (allows sudden spikes up to capacity).
•Leaky Bucket: Enforces a strict, smoothed output processing rate (no bursts).
•Sliding Window Counter: Memory-efficient (stores 2 integer keys in Redis) and eliminates fixed-window 2x boundary spikes.

Common Architectural Pitfall

Using Fixed Window Counters (e.g. 100 req/min); an attacker sends 100 requests at 00:59 and 100 requests at 01:01, pushing 200 req in 2 seconds.

Production Best Practice

Use Sliding Window Counter or Token Bucket in Redis via atomic Lua scripts.

Next: CAP Theorem & PACELC Tradeoffs in Distributed Databases