Idempotency Keys Done Right: Making POST Safe to Retry in FastAPI + Redis

A network timeout does not mean the request failed. It means you do not know. The client retries, the server processes the same payment twice, and now you are refunding a customer and explaining to a regulator why your controls let it happen.

Any mutating endpoint that a client will retry — a charge, a transfer, an order, a payout — needs to be safe to call more than once with the same effect as calling it once. That property is idempotency, and for POST you do not get it for free the way you do with a well-behaved PUT. You have to build it. The mechanism the payment industry has settled on is an Idempotency-Key header: the client generates a unique key per logical operation, sends it with the request, and the server guarantees that a given key produces exactly one side effect no matter how many times it arrives.

This is the full pattern in FastAPI 0.115 and Redis 7 — not the toy version that stores a key and hopes, but the one that handles the two requests racing in at once and the client that reuses a key with a different body. If you are still deciding whether your API should carry this obligation at all, that is a decision worth making deliberately rather than by omission.

What the key actually has to guarantee

Three properties, and skimping on any one of them recreates the bug you are trying to kill:

1. Replay. A retry with the same key and the same body returns the original stored response — same status, same payload — without re-running the side effect.
2. Conflict detection. The same key with a different body is a client error, not a silent overwrite. You return 409 and do nothing.
3. Concurrency safety. Two requests bearing the same key arriving milliseconds apart must not both execute. One wins the lock and processes; the other waits or is told the first is in flight.

The first two are storage problems. The third is the one most homegrown implementations get wrong, because a naive check-then-write has a window between reading “no key yet” and writing the result where a second request slips through. Redis closes that window with a single atomic operation.

The atomic claim: SET NX

The whole design rests on SET key value NX PX ttl. NX means set only if the key does not already exist; the command returns the new value on success and nil if the key was already there. That test-and-set is atomic on the Redis server, so exactly one concurrent request can create the key. In redis-py that is redis.set(name, value, nx=True, px=milliseconds), which returns True on a successful claim and None if the key existed. The winner does the work; every other caller sees the key and branches into replay-or-wait. No Lua, no WATCH/MULTI transaction, no distributed lock library.

We store the key in two phases. On the atomic claim we write a small pending marker holding the request fingerprint — a hash of the body — so a mismatched retry can be rejected even before the original finishes. When the route returns, we overwrite that with the completed record: fingerprint, status code and serialised response body, under a TTL long enough to outlive any sane client retry window (24 hours is the common choice for payments).

The FastAPI dependency

The trick that makes this drop onto any route cleanly is that a FastAPI dependency can read the raw body with await request.body() — the result is cached on the request, so your route handler still parses its Pydantic model normally afterwards. The dependency reads the header, fingerprints the body, runs the claim, and either short-circuits with the stored response or hands control back to the route.

import hashlib
import json

import redis.asyncio as redis
from fastapi import Depends, FastAPI, Header, HTTPException, Request, Response

app = FastAPI()
r = redis.Redis(host="localhost", port=6379, decode_responses=True)

KEY_TTL_MS = 24 * 60 * 60 * 1000  # 24 hours


class Replay(Exception):
    """Raised to short-circuit the route with a stored response."""
    def __init__(self, status_code: int, body: str):
        self.status_code = status_code
        self.body = body


async def idempotency(
    request: Request,
    idempotency_key: str | None = Header(default=None),
) -> str:
    if not idempotency_key:
        raise HTTPException(400, "Idempotency-Key header is required")

    raw = await request.body()  # cached; the route still parses its model
    fingerprint = hashlib.sha256(raw).hexdigest()
    store_key = f"idem:{idempotency_key}"

    # Atomic claim: only the first caller wins.
    claimed = await r.set(
        store_key,
        json.dumps({"state": "pending", "fingerprint": fingerprint}),
        nx=True,
        px=KEY_TTL_MS,
    )
    if claimed:
        return store_key  # we own it; the route runs

    # Key already exists: inspect it.
    record = json.loads(await r.get(store_key))
    if record["fingerprint"] != fingerprint:
        raise HTTPException(409, "Idempotency-Key reused with a different body")
    if record["state"] == "pending":
        # Original is still in flight; tell the client to back off.
        raise HTTPException(409, "A request with this key is already processing")

    raise Replay(record["status_code"], record["response"])


@app.exception_handler(Replay)
async def replay_handler(request: Request, exc: Replay) -> Response:
    return Response(
        content=exc.body,
        status_code=exc.status_code,
        media_type="application/json",
    )


@app.post("/charges")
async def create_charge(payload: dict, store_key: str = Depends(idempotency)):
    # ... perform the real, non-idempotent side effect here ...
    result = {"id": "ch_123", "amount": payload["amount"], "status": "captured"}
    body = json.dumps(result)

    # Persist the completed record so future retries replay it.
    await r.set(
        store_key,
        json.dumps(
            {
                "state": "done",
                "fingerprint": hashlib.sha256(
                    json.dumps(payload, separators=(",", ":")).encode()
                ).hexdigest(),
                "status_code": 201,
                "response": body,
            }
        ),
        px=KEY_TTL_MS,
    )
    return Response(content=body, status_code=201, media_type="application/json")

One caution on the fingerprint: hash the same bytes on both paths. Above, the dependency hashes the raw request body and the route re-hashes a canonicalised dump of the parsed payload, which will differ from the raw bytes if the client sends different whitespace. In production, fingerprint the raw body in both places (thread it through the dependency) or canonicalise consistently — pick one and keep it uniform, or a client retry will look like a conflict.

The failure modes that bite later

The crashed original. If the route dies after claiming the key but before writing the completed record, the pending marker sits there until its TTL expires and every retry gets a 409. That is the safe failure — it blocks a duplicate charge — but you want the pending TTL short (seconds to a couple of minutes) and the completed TTL long, so a genuinely failed request can be retried cleanly without waiting a full day. Set the pending marker with a short px and refresh to the long TTL only on completion.

The side effect that is not in the same transaction. Redis records that you responded; it does not record that the money moved. If your payment write and your Redis write can diverge, the stored response can claim success the ledger never saw. Where correctness is non-negotiable, make the downstream operation itself idempotent on the same key — pass it to the payment provider, whose own idempotency layer is the real backstop — and treat Redis as the fast replay cache, not the source of truth.

Key scope. An idempotency key is meaningful per endpoint and per authenticated caller. Prefix the Redis key with the account or API-key identity so one tenant cannot replay against another’s namespace — the same discipline that keeps API keys from becoming a liability applies to the idempotency namespace.

Why this is a resilience control, not a nicety

Retries are not an edge case. Load balancers retry, client SDKs retry, mobile networks drop and reconnect, and your own queue workers redeliver. An endpoint that double-charges under retry will double-charge in normal operation, not in some rare storm — and the incident, when it lands, shows up in your change-failure rate and your mean time to recovery as surely as any outage. If you measure delivery health properly, the cost of skipping idempotency is already visible in the numbers.

Twenty lines of dependency and one atomic Redis command turn “we do not know if it went through” from an incident into a shrug. Build it into the mutating routes before you have the outage, not after the refund.