resilience • Intermediate

Retry Storms & Full Jitter Exponential Backoff Algorithms

When a database or service experiences an outage, thousands of client retries arrive simultaneously (Retry Storm). Full Jitter randomizes retry intervals across the entire backoff window to prevent synchronized thundering herds.

Intuitive Mental Model

The Traffic Light Outage: If 500 cars wait at a broken traffic light and all accelerate simultaneously the instant the green light turns on, they immediately crash into each other. Staggering acceleration by random intervals lets everyone merge smoothly.

Architecture Blueprint & CodeProduction Standard

// AWS Full Jitter Backoff Algorithm:
function calculateFullJitterBackoff(attempt: number, baseMs = 100, maxMs = 20000): number {
  const exponentialCap = Math.min(maxMs, baseMs * Math.pow(2, attempt));
  // Random integer between 0 and exponentialCap:
  return Math.floor(Math.random() * exponentialCap);
}

Key Architectural Takeaways

•Prevents Thundering Herds: Spreads retry traffic uniformly over time, allowing struggling backends to recover.
•Decorrelated Jitter: Alternative algorithm that uses sleep = min(max, rand(base, sleep * 3)).

Common Architectural Pitfall

Using deterministic exponential backoff (e.g. exactly 1s, 2s, 4s, 8s) without jitter; all clients still hit the server in synchronized waves.

Production Best Practice

Always multiply or randomize backoff with full jitter.

Next: Distributed Tracing & Observability (OpenTelemetry & Jaeger)