Timeouts and Circuit Breakers for Outbound API Calls: Failing Fast Instead of Hanging

One slow third-party dependency can quietly take your whole service down. Not by returning errors — by returning nothing, slowly, until every worker is stuck waiting and there is no capacity left to serve anyone.

This is the failure mode that surprises teams who thought they had resilience covered because they handle exceptions. Exceptions are the easy case: something fails, you catch it, you move on. The dangerous case is the call that never fails and never returns. A payment provider having a bad afternoon, a KYC API under load, an internal service mid-deploy — each of these can leave your outbound request hanging. Do that across enough concurrent requests and you exhaust the worker pool. The upstream degraded; your service went fully dark. The fix is three defensive layers on every outbound call: strict timeouts, bounded retries for the calls that can safely be retried, and a circuit breaker that stops hammering a dependency that is clearly down.

Timeouts do more work than retries

If you add only one thing, add timeouts. A retry policy on a call with no timeout is worse than useless — it just means you now hang on several attempts instead of one. The timeout is what converts an unbounded wait into a bounded, catchable failure, and everything else builds on that.

The trap is the client default. Many HTTP clients ship with either no timeout or a generous one, and people assume the framework has it covered. It usually has not. httpx is better than most here — it applies a five-second default and raises rather than hanging forever — but five seconds per phase is still far too long for a request that sits on a hot path serving live traffic. Worse, a single scalar timeout hides the fact that a request has several distinct phases, each of which can stall for different reasons.

Setting httpx timeouts that actually bite

httpx splits the timeout into four phases, and they fail for different reasons, so they deserve different numbers. Connect is how long you wait to open the socket — a network or DNS problem shows up here, and it should be short. Read is how long you wait for the next chunk of response data, which is where a genuinely slow upstream bites. Write is the send side. Pool is how long you queue for a free connection from the local pool, and setting it low is what stops a backlog of requests silently piling up when the upstream slows down.

Pass an httpx.Timeout object with a sane default and tighter overrides where you can justify them. A two-second connect timeout is generous for any healthy network; if you cannot open a socket in two seconds, waiting longer rarely helps.

Retry only what is safe to retry

Retries are where good intentions cause outages. The rule is narrow and non-negotiable: only retry idempotent requests, and only on failures that a retry could plausibly fix. A GET, PUT or DELETE can be repeated safely by definition. A POST that creates a charge cannot — retrying a request that may have already succeeded but whose response you never saw is how you bill a customer twice. If you need retriable writes, make the endpoint idempotent explicitly with an idempotency key, and only then let the client retry.

Two more constraints. First, retry on transport failures and 5xx responses, never on 4xx — a 400 or 422 means your request was wrong, and it will be wrong every time. Second, back off with jitter. A tight retry loop against a struggling upstream is a self-inflicted denial-of-service; adding randomised delay spreads the load so a recovering service is not knocked straight back over by a synchronised thundering herd. Keep the attempt count low — three total is plenty on a hot path.

The circuit breaker: fail fast, then recover

Timeouts and retries handle a single call. A circuit breaker handles the pattern across calls. When a dependency has failed repeatedly, there is no point letting the next hundred requests each wait out a full timeout before failing — that is precisely how you burn the worker pool. The breaker tracks failures and, once it crosses a threshold, “opens”: subsequent calls fail immediately with a CircuitBreakerError instead of touching the network at all. After a cool-down it allows a single trial request through; if that succeeds, the circuit closes and normal traffic resumes.

pybreaker gives you this in a few lines. The one detail people miss is exclude: a client error must not count towards tripping the breaker, because a 404 tells you nothing about the upstream’s health. Here is the full wrapper — timeouts, idempotent-only retries with jittered backoff, and the breaker sitting outside the retry loop so that repeated failures across separate calls are what open it.

# resilient_client.py
from __future__ import annotations

import random
import time

import httpx
import pybreaker

# Idempotent methods are the only ones we ever retry automatically.
IDEMPOTENT_METHODS = frozenset({"GET", "HEAD", "OPTIONS", "PUT", "DELETE"})

# Aggressive, per-phase timeouts. Nothing is allowed to hang indefinitely.
TIMEOUT = httpx.Timeout(
    timeout=5.0,   # default applied to any phase not overridden below
    connect=2.0,   # give up fast if we cannot open the socket
    read=5.0,      # a slow upstream response is the usual killer
    write=5.0,
    pool=1.0,      # do not queue behind an exhausted connection pool
)


def _is_client_error(exc: BaseException) -> bool:
    # A 4xx means our request was wrong, not that the upstream is unhealthy,
    # so it must not count towards tripping the breaker.
    return (
        isinstance(exc, httpx.HTTPStatusError)
        and exc.response.status_code < 500
    )


breaker = pybreaker.CircuitBreaker(
    fail_max=5,          # consecutive failures before the circuit opens
    reset_timeout=30,    # seconds open before one trial request is allowed
    exclude=[_is_client_error],
    name="payments-upstream",
)

_client = httpx.Client(
    timeout=TIMEOUT,
    limits=httpx.Limits(max_connections=50),
)


def _request_with_retries(
    method: str, url: str, *, max_attempts: int = 3, **kwargs
) -> httpx.Response:
    attempt = 0
    while True:
        attempt += 1
        try:
            response = _client.request(method, url, **kwargs)
            response.raise_for_status()
            return response
        except (httpx.TransportError, httpx.HTTPStatusError) as exc:
            retryable = (
                method in IDEMPOTENT_METHODS
                and attempt < max_attempts
                and not _is_client_error(exc)
            )
            if not retryable:
                raise
            # Full-jitter backoff: sleep in [0, base * 2 ** (attempt - 1)].
            backoff = min(2.0, 0.1 * (2 ** (attempt - 1)))
            time.sleep(random.uniform(0, backoff))


def call_upstream(method: str, url: str, **kwargs) -> httpx.Response:
    # The breaker wraps the whole retry loop: repeated failures across calls
    # trip it, and while open it raises CircuitBreakerError immediately.
    return breaker.call(_request_with_retries, method, url, **kwargs)

Wiring it into FastAPI

One subtlety catches people out. pybreaker and the synchronous httpx.Client are blocking, so calling them directly inside an async def endpoint would block the event loop and stall every other request. The clean fix on FastAPI is to declare the endpoint as a plain def: FastAPI runs synchronous handlers in a worker thread pool, so the blocking work stays off the loop without any manual thread management.

# main.py
import httpx
import pybreaker
from fastapi import FastAPI, HTTPException

from resilient_client import call_upstream

app = FastAPI()


# A sync ("def") endpoint: FastAPI runs it in a worker thread, so the blocking
# httpx.Client and pybreaker calls never stall the event loop.
@app.get("/quote/{symbol}")
def get_quote(symbol: str):
    try:
        response = call_upstream(
            "GET", f"https://prices.example.com/v1/{symbol}"
        )
    except pybreaker.CircuitBreakerError:
        # Circuit is open: fail fast with a clear signal instead of hanging.
        raise HTTPException(status_code=503, detail="Upstream unavailable")
    except httpx.HTTPError:
        raise HTTPException(status_code=502, detail="Upstream error")
    return response.json()

Note the two distinct failure responses. An open circuit returns 503 — “we know this is down, come back shortly” — while a genuine upstream error returns 502. That distinction is worth keeping, because it tells your own monitoring and your callers the difference between a dependency that is failing and one you have deliberately stopped calling.

Picking the numbers

The defaults above are a starting point, not a law. Set the read timeout from the upstream’s real latency, not its advertised one: measure the 99th percentile of successful responses and give it modest headroom. A timeout tighter than normal latency turns healthy calls into failures; one many times looser defeats the point. Size the breaker’s fail_max and reset_timeout to the dependency — a flaky best-effort service can tolerate a higher failure count before you give up on it, while a critical one on the request path should trip early and retry recovery often. And treat every number as instrumented: a breaker that opens is a signal, and how quickly you notice and close it feeds straight into your mean time to recovery.

The uncomfortable part is that none of this is visible until the day an upstream goes slow — and by then it is too late to add it. Timeouts, bounded retries and a breaker are cheap to fit while everything is calm. They are the difference, on the bad afternoon, between one degraded dependency and a full outage of your own making.