AI Incident Response and Serious-Incident Reporting Duties

Most firms have an incident response process built for security breaches and outages. Almost none have one built for AI systems behaving badly — a model producing discriminatory outputs, a system making harmful decisions at scale, an agent taking an action it should not have. As AI moves into consequential decisions and the EU AI Act attaches serious-incident reporting duties to high-risk systems, the gap between “we have incident response” and “we can handle an AI incident and report it” becomes a real exposure.

Why an AI incident is a different kind of incident

A security incident has a familiar shape: something was breached, you contain it, you recover, you report. An AI incident is messier. The system may still be running and doing exactly what it was built to do, while producing harm — biased decisions, unsafe outputs, actions with unintended consequences. There is often no breach to contain, just a behaviour to catch, and catching it requires monitoring the system’s outputs and effects, not just its uptime. The questions are different too: not “how did they get in?” but “what did the model do, to whom, how many times, and why?” — and answering them requires the kind of logging and explainability that AI systems frequently lack.

The reporting duties you may not have mapped

The EU AI Act introduces serious-incident reporting obligations for high-risk AI systems, requiring providers to report certain incidents to authorities within defined timelines. That sits alongside, not instead of, the reporting you may already owe — a GDPR breach if personal data is involved, a DORA operational-incident report if you are a financial entity, a CRA report if the AI is in a product. A single AI incident can trigger multiple reporting duties on multiple clocks to multiple authorities, and a process that only knows how to file a security-breach notification will miss most of them.

Building AI-ready incident response

• Extend detection to behaviour, not just breach. You need to be able to notice that an AI system is producing harmful outputs or decisions, which means monitoring what it does, not only whether it is up.
• Log for reconstruction. When an AI incident occurs, you must be able to reconstruct what the system did and why. If your AI systems do not produce that trail, you cannot investigate or report properly.
• Map AI failures to reporting duties in advance. Know, before an incident, which kinds of AI failure trigger which reports to which authorities on which timelines. Working that out during a live incident is too late.
• Assign ownership. An AI incident falls between the security, data, legal and product functions; someone has to own the AI incident playbook, or it falls through the gaps.

The firms that get this right treat AI incident response as a distinct capability that extends their existing process, wired to the specific reporting duties AI attracts. The ones that do not will discover, in the middle of their first serious AI incident, that their well-drilled breach process does not fit the problem and does not know who to notify.

Who this is for

This reading is for:

• CTOs and compliance leads whose incident process predates AI
• Firms deploying high-risk AI systems into the EU
• Risk owners mapping AI failures onto reporting obligations
• Boards asking what happens when an AI system goes wrong

Sixteen Pillars extends your incident response to AI behaviour and maps AI failures to the specific reporting duties they trigger, before the first serious incident. Pricing is published at /pricing/. If this is live for your organisation and you would like an independent reading, the place to start is a conversation.

Sixteen Pillars is a technology governance consultancy based in Cyprus. Engagements run remote across the EU, UK, and Middle East, with on-site time where the engagement requires it.