Replaying an Agent Run: Event-Sourced Traces for Audit and Reproduction

An agent you cannot replay is an agent you cannot audit. If the only record of what an agent did is a chat transcript and a hope, you have nothing a supervisor will accept.

Most agent observability being sold today logs the wrong thing. It captures the pretty conversation — the user’s prompt, the model’s final answer, maybe a token count for the invoice. What it does not capture is the run: the exact sequence of model calls and tool invocations, the inputs to each, the outputs each returned, and the non-deterministic choices made along the way. When something goes wrong — a bad trade instruction, a customer told something untrue, a file deleted that should not have been — a transcript tells you the outcome. It does not let you reconstruct how the agent got there, and reconstruction is the whole job of an investigation.

The fix is not a better dashboard. It is a different data model. Record every event an agent generates as an append-only log, capture the sources of non-determinism at the moment they occur, and you can replay the run deterministically afterwards — for debugging, for audit, and for the regulator who asks you to prove exactly what your system did on the afternoon of an incident.

Why a transcript is not a trace

An agent is a model wrapped in a harness that lets it call tools, read state and loop. I have written before about why boards should think of an agent as model plus harness; the point that matters here is that the interesting behaviour lives in the harness, not the model. The model proposes; the harness decides what actually runs. So the audit record has to sit at the harness boundary, capturing every crossing: each request sent to the model, each response received, each tool call dispatched and its result, each control-flow decision the loop made.

Treat those crossings as immutable events, appended in order, never edited. This is event sourcing applied to agents: the log is not a byproduct of the run, it is the run. State at any point is a function of the events up to that point, which means you can rebuild the agent’s world at step 14 by replaying steps 0 to 13. Nothing is inferred after the fact and nothing is overwritten, so the record is both complete and tamper-evident.

Capturing the non-determinism

Replay only works if you record the things you cannot recompute. An agent run is riddled with them, and each one is a place where a naive re-run will diverge from the original:

• Model outputs. Language-model sampling is non-deterministic in practice even at temperature zero, and the weights behind an API change without notice. Do not plan to re-call the model on replay — record the exact response it gave and serve that back.
• The model version. Pin the precise model identifier on every call. A label like "the latest one" is not an answer a regulator accepts when the behaviour under investigation depended on a specific build.
• Seeds and randomness. Any sampling seed the provider exposes, and any RNG your own harness uses, must be captured so a re-run makes the same choices.
• Time and external state. Wall-clock timestamps, the contents of a database row, a price feed, a third-party API response — anything the world supplied. Record it, because the world will have moved on by the time you investigate.

The discipline here is the same one that makes retries safe: capture the response to a side-effecting call once and reuse it rather than re-issuing it. It is worth reading the pattern for idempotency keys done right alongside this, because a replay that re-fires a tool call against production is not a replay — it is a second incident.

A trace schema and a replay function

The core is small. An event carries its position, kind, recorded timestamp, payload and provenance, and hashes the previous event so the chain is append-only and tamper-evident. Replay walks the recorded events and hands the agent back exactly what was captured, in order — any divergence from the recorded inputs is raised, because that divergence is precisely what an investigation is hunting for.

"""Event-sourced trace for an agent run.

Every model call, tool invocation and decision is appended to an
immutable log. Non-determinism (model output, seeds, timestamps,
tool responses) is captured at record time so the run can be
replayed deterministically for debugging, audit or reconstruction.
"""
from __future__ import annotations
import json, hashlib, time, uuid
from dataclasses import dataclass, asdict
from typing import Any, Callable


@dataclass(frozen=True)
class Event:
    seq: int                 # monotonic position in the run
    kind: str                # "model_call" | "tool_call" | "decision"
    at: str                  # ISO-8601 wall-clock, recorded not recomputed
    payload: dict[str, Any]  # {"request": ..., "response": ...}
    model: str | None = None # e.g. "claude-opus-4-8" — pin the exact build
    seed: int | None = None  # sampling / RNG seed where one is exposed
    prev_hash: str = ""      # hash of the prior event — append-only chain

    def digest(self) -> str:
        body = json.dumps(asdict(self), sort_keys=True, default=str)
        return hashlib.sha256(body.encode()).hexdigest()


class Trace:
    """Append-only event log for a single agent run."""

    def __init__(self, run_id: str | None = None):
        self.run_id = run_id or str(uuid.uuid4())
        self.events: list[Event] = []

    def append(self, kind, payload, *, model=None, seed=None) -> Event:
        prev = self.events[-1].digest() if self.events else ""
        ev = Event(len(self.events), kind, _now(), payload,
                   model=model, seed=seed, prev_hash=prev)
        self.events.append(ev)
        return ev

    def verify(self) -> bool:
        """Confirm the chain has not been altered after the fact."""
        prev = ""
        for ev in self.events:
            if ev.prev_hash != prev:
                return False
            prev = ev.digest()
        return True


def _now() -> str:
    return time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())


class ReplayDivergence(RuntimeError):
    """Raised when a replay does not match the recorded run."""


def replay(trace: Trace, agent: Callable) -> Any:
    """Re-run the agent, serving recorded outputs instead of live calls.

    The agent asks for a model completion or a tool result; the harness
    returns exactly what was captured, in recorded order. Unexpected or
    drifted calls raise — that is the finding, not a failure.
    """
    pending = iter(trace.events)

    def _next(kind: str, request: Any) -> Any:
        ev = next(pending)
        if ev.kind != kind:
            raise ReplayDivergence(f"expected {kind}, got {ev.kind} @ {ev.seq}")
        if ev.payload["request"] != request:
            raise ReplayDivergence(f"input drift @ seq {ev.seq}")
        return ev.payload["response"]

    model = lambda req: _next("model_call", req)
    tool = lambda req: _next("tool_call", req)
    return agent(trace.events[0].payload["request"], model, tool)

The agent under test takes its initial input and two injected callables — one to reach the model, one to reach tools. In production those callables hit real endpoints and each result is recorded via trace.append. Under replay they are swapped for the ones above, which never touch the network. The same agent code runs both times; only the edges change.

What this buys you with a supervisor

The EU AI Act makes this concrete for anyone operating a high-risk system. Article 12 requires high-risk AI systems to technically allow the automatic recording of events over their lifetime, with logging appropriate to the intended purpose and sufficient to identify situations that present a risk or trigger a substantial modification. Article 19 requires providers to keep those automatically generated logs for a period appropriate to the purpose and, unless other law says otherwise, of at least six months — and where the provider is a financial institution, to keep them as part of the documentation their financial-services rules already demand. A hash-chained event log with deterministic replay is a clean way to meet the substance of both: it records the events, it proves they were not altered, and it lets you reconstruct the run rather than merely list it.

The operational payoff has the same shape as the one that makes any incident tractable. Replay collapses the time from "something went wrong" to "here is the exact step where it went wrong", which is the reconstruction half of mean-time-to-recovery; the same instinct that drives instrumenting the DORA four for a deployment pipeline applies to an agent. And because divergence is raised rather than swallowed, a replay that no longer matches is itself a signal: the model changed under you, a tool’s contract drifted, or someone edited the log.

Build the trace first and the agent second. Retrofitting an audit log onto a system that was never designed to record its own decisions is the point at which most teams discover that the run they most need to explain is the one they cannot reproduce.