Governing AI Agents: A Board and Tech-Function Reading

Agentic AI reached production faster than governance did. In the space of a year, systems that merely suggested became systems that act — booking, buying, executing, and calling other systems without a human confirming each step. The technology outran the accountability model, and boards are now asking a question their existing frameworks cannot answer: when an agent does something wrong, who is responsible, and how would we even know?

Why an agent is a different governance object

A traditional application does what it was coded to do. An agent decides how to achieve a goal, and its path is not fully predictable in advance. That single difference breaks several assumptions baked into most control frameworks. Change governance assumes a human authorises each action. Access control assumes a person behind each credential. Audit assumes a traceable, deterministic sequence. An agent with standing permissions, pursuing a goal across multiple systems, satisfies none of these cleanly — which is why bolting AI onto an existing control set tends to leave gaps exactly where the risk concentrates.

The four questions a board should be able to answer

Governance here is not about slowing agents down; it is about being able to answer, before deployment, questions the board will otherwise face after an incident.

• What can this agent actually do? The scope of its permissions, the systems it can reach, and the actions it can take without human sign-off. Most firms cannot produce this for the agents already running.
• Where is the human checkpoint? Which decisions require confirmation, and which the agent may take alone. The boundary should be a deliberate design choice, not an accident of implementation.
• How is it identified and logged? An agent needs its own identity and a complete, reviewable record of what it did — the non-human-identity problem most IAM programmes never planned for.
• Who owns it? A named accountable person for each agent in production, because “the AI did it” is not an answer a regulator or a court will accept.

The framing that works

The useful move is to treat an agent as you would a new employee with system access: scoped permissions, a probation period under supervision, an audit trail, and a manager accountable for its behaviour. That framing translates a novel technology into a control model boards already understand, and it turns “should we allow agents” into the more productive “under what controls.” The EU AI Act’s transparency and, for higher-risk uses, oversight obligations sharpen this — but the governance case stands on operational risk alone, before any regulation applies.

Who this is for

This reading is for:

• Boards asking who is accountable when an autonomous agent acts
• CTOs deploying agentic AI into production workflows
• Risk and compliance leads whose frameworks assume a human in the loop
• Heads of engineering setting guardrails for agent permissions

Sixteen Pillars helps boards and technology functions put a real control model around agents in production, from scoped permissions to named accountability. Pricing is published at /pricing/. If this is live for your organisation and you would like an independent reading, the place to start is a conversation.

Sixteen Pillars is a technology governance consultancy based in Cyprus. Engagements run remote across the EU, UK, and Middle East, with on-site time where the engagement requires it.