This Fixes: Clients Hammering Your API After a 429 Because You Forgot Retry-After

Your API starts shedding load, returns a clean 429, and the incident gets worse in the next second. The clients you just told to slow down retry immediately, in lockstep, because a bare 429 gives them nothing to wait on.

This is one of the most common self-inflicted outages I see in production. The rate limiter is working exactly as designed — it is protecting the service by rejecting excess requests. But a 429 Too Many Requests with no Retry-After header is a rejection with no instruction. The client library sees a failure, treats it as retryable, and fires again straight away. Multiply that by every client hitting the wall at the same moment and you have a retry storm: a synchronised wave of traffic that keeps the service pinned down long after it should have recovered. The fix has two halves, and you need both.

Why a bare 429 makes the overload worse

A 429 says “not now”. It does not, on its own, say “not for another eight seconds”. RFC 9110 defines the Retry-After header precisely for this: it is a hint, valid on both 429 and 503 responses, telling the client how long to wait before trying again. It carries either a delay in seconds (Retry-After: 8) or an HTTP-date. Without it, every well-intentioned retry library falls back to its own default — and the naive default is to retry immediately, or on a fixed one-second timer.

Fixed timers are the trap. If a hundred clients all get throttled in the same window and all retry one second later, you have not spread the load — you have scheduled a hundred simultaneous retries for one second in the future. This is the thundering herd, and it is why the service that returned a correct 429 still falls over. The server knows exactly when its window resets. The only failure is that it did not tell anyone.

The server side: set Retry-After on everything you throttle or shed

Here is a fixed-window limiter for Express 5 that advertises the budget on every response and attaches Retry-After to the rejection. The bucket store is in-memory for clarity; in production this belongs in Redis so it is shared across instances.

// rate-limit.js — Express 5, Node 22
// Attach Retry-After (and RateLimit hints) to throttled responses.

const WINDOW_MS = 60_000;   // fixed window
const MAX_HITS = 100;       // requests per window per key
const buckets = new Map();  // key -> { count, resetAt }

function keyFor(req) {
  // Prefer an authenticated identifier; fall back to client IP.
  return req.get('x-api-key') || req.ip;
}

export function rateLimit(req, res, next) {
  const now = Date.now();
  const key = keyFor(req);
  let bucket = buckets.get(key);

  if (!bucket || now >= bucket.resetAt) {
    bucket = { count: 0, resetAt: now + WINDOW_MS };
    buckets.set(key, bucket);
  }

  bucket.count += 1;
  const remaining = Math.max(0, MAX_HITS - bucket.count);
  const resetSeconds = Math.ceil((bucket.resetAt - now) / 1000);

  // Advertise the budget on every response, not only the rejection.
  res.set('RateLimit-Limit', String(MAX_HITS));
  res.set('RateLimit-Remaining', String(remaining));
  res.set('RateLimit-Reset', String(resetSeconds));

  if (bucket.count > MAX_HITS) {
    // The one header that stops the retry storm.
    res.set('Retry-After', String(resetSeconds));
    return res
      .status(429)
      .json({ error: 'rate_limited', retry_after: resetSeconds });
  }

  next();
}

The same discipline applies when you shed load rather than rate-limit it. If a circuit breaker trips or a queue is full and you return 503 Service Unavailable, set Retry-After on that too — a sensible constant if you cannot compute a precise reset. Any response that says “come back later” must say when. The RateLimit-Remaining and RateLimit-Reset fields shown here follow the pattern many APIs already emit and the IETF RateLimit-headers draft is standardising; they let a well-behaved client slow down before it hits the wall, which is better than any recovery after it.

If you scope limits per API key rather than per IP, the same key you rate-limit on is the one you should be hashing and scoping properly. That is a separate discipline covered in hashing, prefixes, scoping and rotation for API keys.

The client side: back off with full jitter, respect the hint

A server hint is only half the fix. Your own outbound clients — the code that calls other people’s APIs — need to back off correctly whether or not the upstream sets Retry-After. Here is a fetch wrapper on Node 22’s built-in fetch that honours the header when present and falls back to exponential backoff with full jitter when it is absent.

// fetch-retry.js — Node 22 global fetch, backoff with full jitter.

const RETRYABLE = new Set([429, 502, 503, 504]);
const BASE_MS = 200;
const CAP_MS = 20_000;
const MAX_ATTEMPTS = 5;

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

// Parse Retry-After: delta-seconds or an HTTP-date (RFC 9110).
function retryAfterMs(res) {
  const header = res.headers.get('retry-after');
  if (!header) return null;
  const seconds = Number(header);
  if (Number.isFinite(seconds)) return seconds * 1000;
  const when = Date.parse(header);
  return Number.isNaN(when) ? null : Math.max(0, when - Date.now());
}

export async function fetchWithRetry(url, options = {}, attempt = 0) {
  const res = await fetch(url, options);

  if (!RETRYABLE.has(res.status) || attempt >= MAX_ATTEMPTS) {
    return res;
  }

  // Full jitter: a random point in [0, min(cap, base * 2^attempt)].
  const ceiling = Math.min(CAP_MS, BASE_MS * 2 ** attempt);
  const backoff = Math.random() * ceiling;

  // Honour the server's hint if it gave one; otherwise back off ourselves.
  const hinted = retryAfterMs(res);
  const wait = hinted ?? backoff;

  await sleep(wait);
  return fetchWithRetry(url, options, attempt + 1);
}

Two details matter. First, the retry set is deliberately narrow: 429, 502, 503, 504. A 400 or a 403 is not going to succeed on retry, and retrying it just wastes the budget you are trying to protect. Second, retries are only safe on idempotent operations. Replaying a GET is harmless; replaying a POST can double-charge a customer or create a duplicate record unless the endpoint is built to tolerate it. If you retry writes, you need idempotency keys underneath, or this wrapper becomes a machine for producing duplicates.

Why full jitter, and not plain exponential backoff

Plain exponential backoff — wait 200ms, then 400ms, then 800ms — solves the wrong problem. It spaces out one client’s retries, but if every client started at the same moment they all still retry in the same slots, just wider-spaced slots. The peaks are lower but they are still peaks. AWS’s often-cited analysis of this showed that the change which actually flattens the load is not longer waits, it is randomised ones.

Full jitter is the version that works: instead of sleeping for the full computed interval, you sleep for a random duration between zero and that interval. The line in the code above is the whole idea — Math.random() * min(cap, base * 2^attempt). The exponential term keeps the ceiling growing so a struggling service gets progressively more breathing room; the randomisation smears each cohort of clients across the whole window so they stop arriving as a wall. The cap stops the intervals growing without bound. This is the difference between a herd and a trickle.

What to check before you ship

Grep your own codebase for handlers that return 429 or 503 and confirm every one sets Retry-After. Then look at your outbound HTTP clients and confirm they read it, cap their attempts, and jitter their waits — many popular libraries do plain backoff by default and need explicit configuration. Load-test it: hammer the service past its limit and watch whether the retry traffic arrives as a smooth line or a sawtooth. If you instrument retry rate and time-to-recover as first-class signals — the kind of operational metrics behind the DORA four — a retry storm shows up as a recovery that never quite arrives.

A rate limiter without Retry-After is a wall with no sign on it. You built the wall to protect the service; the sign is what stops the clients running into it in formation.