Accuracy, Robustness and Cybersecurity: The AI Act Article 15 Test Battery You Have to Evidence

Article 15 is the point where model evaluation stops being an internal engineering nicety and becomes something a supervisor can demand to see. A single headline accuracy figure will not survive that conversation.

Most teams building high-risk AI systems already measure accuracy somewhere — in a notebook, on a dashboard, in the head of the data scientist who trained the model. Article 15 of the EU AI Act asks for something harder: a defined, repeatable test battery covering accuracy, robustness and cybersecurity, with declared metrics and results you can put in front of a market surveillance authority. This is a reading of what that battery has to contain and how to wire it into the evaluation pipeline so the evidence produces itself rather than being reconstructed under pressure.

The timing has moved, so it is worth being precise. Under the Digital Omnibus adopted by the Council in June 2026, application of the high-risk obligations for stand-alone systems listed in Annex III is now set for 2 December 2027, and for systems embedded in regulated products under Annex I for 2 August 2028. That is more runway than the original 2026 date, not a reprieve. Building the Article 15 battery is a multi-quarter piece of engineering, not a documentation exercise you bolt on at the end.

What Article 15 actually requires

The article is short and its demands are specific. A high-risk system must achieve an appropriate level of accuracy, robustness and cybersecurity, and perform consistently in those respects across its lifecycle. Two obligations sit underneath that. The levels of accuracy and the relevant accuracy metrics must be declared in the instructions for use — you have to name the metric and the number, in writing, for the deployer. And the system must be resilient both to errors and inconsistencies in operation and to deliberate attack.

Article 15 also anticipates benchmarks and measurement methodologies developed with metrology and benchmarking bodies. Those are still emerging; the harmonised standards being drafted through CEN-CENELEC are not yet in a position to hand you a presumption of conformity for evaluation. Treat that as provisional and verify the position when you build. In the meantime the honest answer is that you define defensible metrics and thresholds yourself, and you write down why they are defensible.

“Accuracy” is a metric set, not a number

The most common way to fail Article 15 on paper is to declare a single overall accuracy figure. On an imbalanced problem — fraud, adverse-event detection, most of the interesting cases — an overall figure of 98 per cent can hide a recall of 40 per cent on the class that actually matters. A supervisor who understands the domain will ask exactly that question, and “we reported the headline number” is not an answer.

What you declare should reflect the decision the system supports. That usually means per-class precision and recall, a macro-averaged score that does not let the majority class dominate, and performance broken down across the population segments the system will see in production. It also means naming the operating point — the threshold at which the numbers hold — and the reference dataset they were measured on. An accuracy figure without its operating point and its population is a number without meaning, and Article 15(3) is asking for meaning.

Robustness is what you test on purpose

Robustness under Article 15 is resilience to errors, faults and inconsistencies, including those arising from interaction with people and other systems. In practice that is a set of tests you run deliberately, not a property you hope for. Perturb the inputs within a bounded range and measure how far accuracy falls. Shift the input distribution to approximate the drift you expect between training and deployment. Feed malformed, missing and out-of-range values and confirm the system degrades safely rather than returning a confident wrong answer.

The article explicitly permits robustness to be achieved through redundancy, including backup or fail-safe plans, so the test battery should evidence that the fail-safe fires when the model is out of its depth. For systems that keep learning after deployment, Article 15(4) names feedback loops — biased outputs quietly influencing future inputs — as a risk you have to address. That is a lifecycle concern, which is where the evaluation battery and post-market monitoring stop being separate activities and start feeding each other.

Cybersecurity: the attacks Article 15(5) names by hand

Unusually for a regulation, Article 15(5) lists the AI-specific attacks it wants addressed where appropriate: data poisoning of the training set, model poisoning of pre-trained components, adversarial examples designed to make the model err (also called model evasion), and confidentiality attacks. This is not generic IT security. Your existing controls for network and access do nothing about an adversarial example crafted to flip a classification, and nothing about a membership-inference probe that recovers whether a record was in the training data.

The evidence a supervisor will want is that you tested for these deliberately. Adversarial accuracy under a crafted perturbation, not just random noise. Provenance and integrity controls on the training data and on any pre-trained components you import. An assessment of confidentiality exposure for models trained on personal or sensitive data. You will not defeat every attack, and Article 15 does not ask you to. It asks you to show you engaged with the named threats and made proportionate decisions about them.

Acceptance criteria and the record

The discipline that turns this into a compliance artefact is setting acceptance criteria before the run, not reading them off the result afterwards. A minimum clean accuracy, a minimum macro score, a maximum tolerated drop under perturbation — declared in advance and tied back to the risk the system carries. This is where Article 15 connects to the Article 9 risk management system: the acceptance thresholds are risk decisions, and they belong in the risk record as well as the evaluation log.

The output of a run should be a structured record, not a screenshot. Dataset version, sample count, each metric, each threshold, and a pass or fail against it — emitted automatically so it flows into the technical documentation the Act requires. A minimal harness looks like this.

"""Article 15 evaluation harness (illustrative).

Computes clean and adversarial metrics for a high-risk classifier and
emits a structured record for the technical file. Framework-agnostic:
supply predict_proba(X) returning an (n, n_classes) array of scores,
plus the labelled test set.
"""
from __future__ import annotations
import json
import time
import numpy as np


def macro_f1(y_true, y_pred, n_classes):
    scores = []
    for c in range(n_classes):
        tp = np.sum((y_pred == c) & (y_true == c))
        fp = np.sum((y_pred == c) & (y_true != c))
        fn = np.sum((y_pred != c) & (y_true == c))
        denom = 2 * tp + fp + fn
        scores.append((2 * tp / denom) if denom else 0.0)
    return float(np.mean(scores))


def perturb(X, epsilon, rng):
    # Bounded L-infinity noise: a cheap stress test, not a substitute for a
    # gradient-based attack where the model exposes gradients.
    return X + rng.uniform(-epsilon, epsilon, size=X.shape)


def evaluate(predict_proba, X, y, n_classes, thresholds, epsilon=0.03, seed=7):
    rng = np.random.default_rng(seed)
    y = np.asarray(y)

    clean_pred = predict_proba(X).argmax(axis=1)
    clean_acc = float(np.mean(clean_pred == y))
    clean_f1 = macro_f1(y, clean_pred, n_classes)

    adv_pred = predict_proba(perturb(X, epsilon, rng)).argmax(axis=1)
    adv_acc = float(np.mean(adv_pred == y))
    robustness_drop = clean_acc - adv_acc

    record = {
        "timestamp_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "n_samples": int(len(y)),
        "epsilon": epsilon,
        "metrics": {
            "clean_accuracy": round(clean_acc, 4),
            "macro_f1": round(clean_f1, 4),
            "adversarial_accuracy": round(adv_acc, 4),
            "robustness_drop": round(robustness_drop, 4),
        },
        "acceptance": {
            "clean_accuracy": clean_acc >= thresholds["min_accuracy"],
            "macro_f1": clean_f1 >= thresholds["min_macro_f1"],
            "robustness_drop": robustness_drop <= thresholds["max_drop"],
        },
    }
    record["passed"] = all(record["acceptance"].values())
    return record


if __name__ == "__main__":
    # Thresholds are risk decisions: declare them before the run, not after.
    thresholds = {"min_accuracy": 0.92, "min_macro_f1": 0.88, "max_drop": 0.05}
    # `model` is supplied by your pipeline and must expose predict_proba.
    result = evaluate(model.predict_proba, X_test, y_test,
                      n_classes=4, thresholds=thresholds)
    print(json.dumps(result, indent=2))

The perturbation here is deliberately crude — bounded random noise as a first-pass stress test. For a model that exposes gradients you would replace it with a proper adversarial method, and for a language model the analogue is a curated set of jailbreak and prompt-injection cases scored the same way. The shape is what matters: a callable, declared thresholds, a machine-readable verdict.

What a supervisor will actually ask

The question will not be “is your model accurate.” It will be: show us the metrics you declared in the instructions for use, the dataset you measured them on, the thresholds you set and why, the adversarial tests you ran, and the last time this ran green. If the answer to all of that is one number a data scientist remembers, you do not have an Article 15 position — you have a demo. The battery is the difference between the two, and it has to exist before the auditor does.