Redis Cache Stampede — Singleflight

Homepage product JSON lives in Redis under product:42 with a 60-second TTL. At T+60.001, two thousand concurrent requests miss. Two thousand processes hit Postgres with the same query. CPU spikes, p99 explodes, Redis refills slowly with stampeded writes. That is a cache stampede (thundering herd). The key was hot; expiry was synchronized; nobody coordinated the refill.

What singleflight buys you

Singleflight (also called request coalescing): for a given key, only one in-flight loader runs; everyone else waits for that result. Go’s golang.org/x/sync/singleflight is the namesake. In Node you can do it in-process; across many app instances you need a Redis lock or similar.

In-process coalescing

const inflight = new Map<string, Promise<unknown>>();

export async function getOrLoad<T>(
  key: string,
  load: () => Promise<T>,
): Promise<T> {
  const hit = await redis.get(key);
  if (hit) return JSON.parse(hit) as T;

  let p = inflight.get(key) as Promise<T> | undefined;
  if (!p) {
    p = (async () => {
      try {
        const value = await load();
        await redis.set(key, JSON.stringify(value), "EX", 60);
        return value;
      } finally {
        inflight.delete(key);
      }
    })();
    inflight.set(key, p);
  }
  return p;
}

This protects one Node process. Ten Kubernetes pods still stampede ten loaders unless you add a distributed gate.

Distributed lock around refill

async function getOrLoadDistributed<T>(key: string, load: () => Promise<T>) {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached) as T;

  const token = crypto.randomUUID();
  const locked = await redis.set(`lock:${key}`, token, "NX", "EX", 5);
  if (locked) {
    try {
      const value = await load();
      await redis.set(key, JSON.stringify(value), "EX", 60);
      return value;
    } finally {
      // delete lock only if we still own it (Lua compare-and-del in production)
      await redis.del(`lock:${key}`);
    }
  }

  // someone else is loading — brief wait then re-read
  await new Promise((r) => setTimeout(r, 50));
  const again = await redis.get(key);
  if (again) return JSON.parse(again) as T;
  return load(); // fallback if lock holder failed
}

Use a proper Redlock/Lua unlock in production; the sketch shows the control flow.

Soft TTL / early refresh

Hard expiry makes every client discover the miss at once. Alternatives:

Store { value, staleAt, hardExpireAt }. After staleAt, one requester refreshes while others still get the stale value.
Probabilistic early expiration: refresh with probability rising as TTL approaches zero (XFetch-style) so refreshes smear across time.
Separate short negative-cache TTL for misses so you do not stampede empty results.

Probabilistic vs lock

Approach	Pros	Cons
In-process singleflight	Simple, fast	Per instance only
Redis NX lock	Cross-instance	Lock failure / deadlock care
Soft TTL serve-stale	Best UX under load	Need tolerance for staleness
No cache on miss path	—	Guaranteed DB melt

Failure modes

Lock holder crashes — TTL on the lock (5s above) is mandatory.
Loader is slow — Waiters time out and all fall through to DB; add a max wait and a degraded response.
Huge values — Coalescing still helps DB, but Redis SET of multi-MB JSON can block; compress or cache pointers.
Many keys expire together — Deploy that sets the same TTL on warm-up creates synchronized expiry waves; add jitter to TTL: 60 + random(0, 10).

How to see it in metrics

Spike in DB QPS aligned with Redis misses for one key. Cache hit ratio cliff at TTL boundaries. Latency histogram with a sharp ridge every N seconds. Log cache_miss_load_started with a counter per key — if it exceeds 1 per key per window, your coalescing is not working across the fleet.

Stampede prevention is not “add Redis.” You already have Redis. It is making sure one worker pays for the miss while the rest reuse the result — in-process first, distributed when you scale out, soft TTL when staleness is acceptable.

Tools

Redis Cache Stampede — Singleflight Pattern