Post-Market Monitoring for High-Risk AI: The Plan, the Signals and the Feedback Loop

A post-market monitoring plan you can hand to a notified body is worth nothing if the telemetry it describes is not actually running in production. Article 72 turns a document into an operating loop, and most teams have built only the document.

Post-market monitoring is the part of the AI Act that reads like paperwork and behaves like engineering. Article 72 requires providers of high-risk systems to establish and document a post-market monitoring system, and to base it on a plan that forms part of the technical documentation. The wording is bureaucratic; the obligation is not. What it substantively asks is that you collect and analyse performance data across the deployed lifetime of the system, and that you can show a supervisor the system still meets the requirements it was certified against. That is a data-engineering problem wearing a compliance badge, and it is the reason a plan drafted by the compliance team in isolation tends to describe a loop nobody is running.

What Article 72 actually asks for

Strip the article back and it makes four demands. Establish and document a post-market monitoring system, proportionate to the nature of the technology and the risks. Actively and systematically collect, document and analyse relevant data on the system’s performance throughout its lifetime. Base that activity on a monitoring plan that lives inside the technical documentation. And, for the credit-scoring and certain insurance systems run by financial institutions, integrate the monitoring into the arrangements those firms already operate under sectoral governance rather than standing up a parallel process.

Two dates matter here, and both need care. The Commission was mandated under Article 72(3) to publish a template for the monitoring plan by 2 February 2026 through an implementing act; build your plan to whatever version of that template is current when you read this rather than to a draft. And the application date for stand-alone high-risk systems in Annex III, originally 2 August 2026, is expected to move to 2 December 2027 under the provisional Digital Omnibus agreement reached in May 2026. Treat that later date as provisional until it appears in the Official Journal, because it is not law until it does.

A plan without telemetry is a promise

The plan has to name the signals it depends on, and those signals have to exist as instrumented, retained data before the plan means anything. In practice a workable monitoring loop for a high-risk model collects at least the following:

• Input distributions. The statistical shape of the features going in, so you can detect when the population the model sees in production diverges from the one it was trained and validated on.
• Output distributions. The shape of the scores or classifications coming out, which shifts before performance visibly degrades and often before anyone complains.
• Outcomes, where you can get them. The ground truth that eventually confirms whether a prediction was right. Labels usually arrive late, so build the store to join a prediction made today to an outcome recorded months later.
• Human-override rate. How often the people exercising oversight reject or amend the system’s output. A rising override rate is one of the earliest honest signals that trust in the model is slipping.
• Operational health. Latency, availability, feature-pipeline failures and data-quality faults at ingress, because a model fed stale or partial inputs fails quietly rather than loudly.

Most of this you are probably already emitting somewhere for reliability reasons. The gap is rarely that the data does not exist; it is that it is scattered across observability tools built for uptime, never joined to model identity, and never retained long enough to reconstruct what the system was doing when a question is asked about it. The automatic logging Article 12 already requires gives you a good deal of the raw material; post-market monitoring is what you do with it once it is aggregated rather than kept per-event.

Detecting drift and degradation

The core detection job is comparing a live distribution against a fixed reference baseline and quantifying how far it has moved. The Population Stability Index is a plain, defensible way to do that for a single feature or score, and it produces a number you can attach a threshold to and explain to a non-specialist. The monitor below computes it, grades the result against two thresholds recorded in the plan, and emits a structured alert only when something has actually moved.

"""Population Stability Index (PSI) drift monitor for a high-risk AI system.
Compares a live feature or score distribution against a fixed reference
baseline and emits a structured alert into the post-market monitoring sink.
"""
from __future__ import annotations
import json, datetime as dt
import numpy as np

# Review thresholds agreed with the risk owner and recorded in the plan.
PSI_WARN = 0.10   # investigate at the next scheduled review
PSI_TRIP = 0.25   # trigger a corrective-action review out of cycle

def psi(reference: np.ndarray, live: np.ndarray, bins: int = 10) -> float:
    """Return the PSI between a reference and a live sample."""
    edges = np.quantile(reference, np.linspace(0, 1, bins + 1))
    edges[0], edges[-1] = -np.inf, np.inf
    ref = np.histogram(reference, edges)[0] / len(reference)
    cur = np.histogram(live, edges)[0] / len(live)
    ref = np.clip(ref, 1e-6, None)          # avoid division by zero
    cur = np.clip(cur, 1e-6, None)
    return float(np.sum((cur - ref) * np.log(cur / ref)))

def evaluate(system_id: str, feature: str,
             reference: np.ndarray, live: np.ndarray) -> dict:
    score = psi(reference, live)
    status = "ok"
    if score >= PSI_TRIP:
        status = "trip"
    elif score >= PSI_WARN:
        status = "warn"
    return {
        "system_id": system_id,
        "feature": feature,
        "metric": "psi",
        "value": round(score, 4),
        "status": status,
        "sample_size": int(len(live)),
        "observed_at": dt.datetime.now(dt.timezone.utc).isoformat(),
    }

if __name__ == "__main__":
    alert = evaluate(
        system_id="credit-scoring-v3",
        feature="applicant_income",
        reference=np.load("baseline_income.npy"),
        live=np.load("last_24h_income.npy"),
    )
    # Emit one JSON line per evaluation for the dashboard to ingest.
    if alert["status"] != "ok":
        print(json.dumps(alert))

PSI is a starting point, not the whole battery. Run the same pattern on the output distribution, add a performance metric wherever outcomes let you compute one, and track the override rate as its own series. The point is not the specific statistic; it is that every signal in the plan has a monitor behind it, a threshold attached to it, and an alert that goes somewhere a human will see it.

Thresholds, cadence and the feedback loop

A threshold is only useful if crossing it does something. The plan should define two responses. A warn-level breach feeds the scheduled review — monthly or quarterly, proportionate to the risk — where the accumulated signals are read together and a named owner decides whether the system still meets the requirements it was placed on the market against. A trip-level breach pulls a review forward out of cycle, because the distribution has moved far enough that waiting for the calendar is itself a decision you would struggle to defend. Both feed the same destination: a documented corrective action, which might be retraining, a threshold recalibration, a scope restriction, or in the sharpest cases taking the system out of service.

This is the loop closing. Field signal to detection, detection to review, review to corrective action, corrective action back into the model — and the whole trace retained as the evidence that ongoing conformity is something you operate rather than something you asserted once at certification. It is also where monitoring meets your Article 9 risk management system, which is meant to run continuously across the lifecycle; post-market monitoring is the sensor array that keeps that risk assessment honest instead of frozen at launch.

Monitoring also sits directly upstream of Article 73. Where a signal reveals a serious incident, the reporting clock is short — generally no later than fifteen days after you become aware, ten days where a death is involved, and two days for a widespread infringement. A monitoring loop that surfaces the incident weeks late does not just miss the problem; it starts you inside a reporting window you have already half spent.

Who has to own this

The plan is a compliance artefact; the loop is an engineering system, and the two fail in different ways. A plan with no telemetry behind it is a promise you cannot keep. Telemetry with no plan over it is a dashboard nobody is accountable for. The work is to make sure every signal named in the document is instrumented, retained and thresholded, and that a named person acts when a threshold trips. That also means knowing precisely which systems carry the obligation in the first place — a question the AI system inventory mapped to the risk tiers is meant to answer before any of this monitoring gets built.

Build the loop now, against a real production estate, and the plan writes itself from something that is already running. Write the plan first and wire the telemetry later, and you will be reverse-engineering a document into a system under a deadline, which is the most expensive order to do this in.