caching-queues • Advanced

Caching Strategies & Thundering Herd (Cache Stampede) Defense

Caching patterns (Cache-Aside, Write-Through, Write-Behind) optimize latency. Cache Stampedes occur when high-traffic keys expire, causing 10,000 concurrent database queries; resolved via Distributed Mutexes or Probabilistic Early Expiration (XFetch).

Intuitive Mental Model

The Supermarket Free Samples Table: When the sample plate empties, 500 shoppers do not all rush into the kitchen at once; one store clerk locks the kitchen door, prepares a fresh tray, and restocks the table.

Architecture Blueprint & CodeProduction Standard

function shouldRecomputeEarly(expiry: number, delta: number, beta: number = 1.0): boolean {
  const now = Date.now();
  const timeRemaining = expiry - now;
  const probabilisticThreshold = -beta * delta * Math.log(Math.random());
  return probabilisticThreshold > timeRemaining;
}

Key Architectural Takeaways

•Cache-Aside (Lazy Loading): Application reads cache first; on miss, queries DB and populates cache.
•Write-Through: Writes data to cache and database synchronously.
•Write-Behind (Write-Back): Writes to cache immediately and flushes to database asynchronously in batches.
•Thundering Herd Defense: Use Redis distributed locks (Redlock) or Probabilistic Early Expiration (XFetch).

Common Architectural Pitfall

Setting identical fixed TTLs (e.g. exactly 3600 seconds) on 1,000,000 cached records, causing simultaneous mass expiration and database collapse.

Production Best Practice

Add random jitter to TTLs: TTL = baseTTL + random(0, 300).

Next: Message Queues: Kafka Partitions, Consumer Groups & DLQ