1. From verifying access to verifying action
For a decade, enterprise security has run on one principle: never trust, always verify. No user, device or system gets automatic access just because it sits inside the network. That principle still holds. What has changed is the object of verification.
The old question was whether a user or system could access a resource. AI adds a much harder one. Can this user, this agent, this model and this chain of tools perform this specific action, on this specific data, right now? Agents act without a person approving each step, models draw on live company data, and retrieval systems hand documents straight to a decision. None of the original architecture assumed any of this.
Authentication tells you who or what is acting. It tells you nothing about whether the action deserves to be trusted.
Most organisations still manage AI adoption on the first question alone, because it is the easy one to tick off. The second question is where AI adoption either succeeds or quietly goes wrong.
2. Who is acting, and who answers for it
Enterprise identity was built for predictable actors: people, applications, service accounts and workloads. An agent can act on behalf of an employee, use its own credentials, call other systems and delegate work to other agents, all without a person approving each step. When something goes wrong, "the AI did it" tells you almost nothing.
Three questions every agent action must answer
- Who initiated the task? Which person, process or system set the objective?
- Which agent executed it? The specific agent identity, not just the application.
- What authority had been delegated to it, and under whose name?
Every production agent needs an attributable identity, but identity alone is not accountability. Authentication tells you who is acting, authorisation tells you what it may do, and accountability tells you who answers for the outcome. These three should not collapse into one just because an agent acted "on behalf of" an authorised employee. An agent can execute an action. It cannot own the consequence.
3. Least privilege becomes least agency
Least privilege gives a person or system only the access it needs. With agents, access is only half the question. Picture an agent connected to email, the CRM, documents, ticketing and finance. Each permission looks reasonable on its own. Combined with software that can reason, choose tools and take several steps, the question becomes how far it can go with what it can reach.
An agent allowed to read an invoice is not automatically an agent that should change supplier details. Same identity, same system, very different agency. If nobody decides how much independent action to delegate, the boundary still gets set, only by accident, through whatever permissions and tools the agent happens to inherit.
Ask what the maximum consequence is that this agent can create before a person has to intervene.
4. Treat content as untrusted, including your own data
Even a well-scoped agent can be tricked into misusing access it legitimately holds. The serious version of prompt injection is indirect. A malicious instruction hides inside an email, a web page or a document the agent was asked to read. You ask for a summary of an email, and the email tells the agent to search for confidential pricing and include it in the reply. Being authorised to read something must never mean trusting the instructions inside it.
Retrieval-augmented generation (RAG) raises the stakes, because the content an agent reads is often your own company data. That creates a trust chain where every step needs its own check.
Authorisation has to happen before data reaches the model, not after. If an employee cannot open a confidential document directly, asking an assistant must not become a side door to it. RAG is therefore a data governance decision made in real time, and AI makes weak permissions far easier to find, because people no longer need to know where information lives. They can simply ask.
5. When AI stops answering and starts acting
A chatbot can give a wrong answer. An agent can take that wrong answer and send an email, update a customer record, run code or change supplier bank details. The model may be the same; the consequence is not. That is why AI intelligence and AI authority need to be separated. The agent proposes, policy decides whether the action is allowed, and for high-impact actions a person approves before anything changes.
The closer AI gets to a real-world consequence, the stronger the verification should become.
Verification also has to keep pace with the task. "Resolve this supplier issue" may become a sequence of steps, from reading an email to retrieving an invoice to sending a reply externally, and the risk rises as the agent moves. One authorisation decision at the start is no longer enough. Authorisation has to travel with the workflow and ask, before each meaningful step, whether this action is still appropriate for this task, this data and this moment. That does not mean a person approving every click. It means policies that recognise when the risk has materially changed.
6. Seeing it, and proving it
If decisions happen throughout an agent's journey, you need to be able to reconstruct that journey afterwards. When a customer receives an incorrect refund issued by an agent, "which user clicked the button?" no longer helps, because there may have been no button and no single decision.
The point is not to record every internal model thought. It is to answer one basic question: how did this business action happen? Observability also makes autonomy measurable, because you can see how an agent behaves with the authority it already has before giving it more.
Regulation turns this into an evidence question. The EU AI Act expects high-risk systems to maintain accuracy, robustness and cybersecurity throughout their lifecycle, and GDPR applies as soon as personal data enters an AI workflow. It is not enough to have a control. You need to show it existed and worked when it mattered: who acted, what data was used, where human oversight happened and what had changed.
7. What changes underneath you
A traditional application behaves the same until someone deploys new code. An AI application depends on a model, system prompts, a knowledge base, plugins, external APIs and orchestration frameworks, and any of them can change independently. The model provider ships a new version, a new document enters the knowledge base, a tool is connected. Nobody changed "the application", yet its behaviour and risk may have.
So what did you actually approve? Not the agent in the abstract, but the specific combination of model, data, instructions, tools and permissions that existed when it was tested. An inventory entry that says "Customer service agent: approved" is not enough. Know which model and version is running, which sources can influence it, which tools it can call, what has changed since the last review, and whether you can roll back.
8. What a Zero Trust architecture for AI looks like
Put the pieces together and one point stands out: the model is not the system. An enterprise AI service is a chain, and each connection raises a different trust question.
Five control areas
- Identity. Who is acting: a person, an agent, a service or another agent?
- Data. What may enter the AI's context, and under whose permissions?
- Policy. Is this action allowed, given the user, agent, data, tool and current risk?
- Execution. What can create a consequence: APIs, code, email, payments, workflows?
- Observability. Can we reconstruct the full path from request to outcome?
The model sits inside this architecture. It should never be the security boundary, because models are probabilistic and can misread context or be manipulated. A stronger design assumes the model can be wrong and the architecture still holds.
No single framework covers all of this, and that is fine. Each one answers a different question, and the goal is a control model where they complement each other rather than five versions of the same control.
| If you need | Start with |
|---|---|
| An enterprise structure for AI risk | NIST AI Risk Management Framework |
| Maturity across existing security capabilities | CISA Zero Trust Maturity Model |
| The attacker's view of AI systems | MITRE ATLAS and the OWASP guidance for LLM applications |
| AI-specific risks across data, models and applications | Google Secure AI Framework (SAIF) |
| Detailed control objectives and mappings | Cloud Security Alliance AI Controls Matrix |
| An AI management system with clear accountability | ISO/IEC 42001 |
9. Where to start, and how much autonomy to allow
Start with the security foundations AI is about to inherit, not with a new AI security tool. If permissions are already too broad, AI does not create the problem; it makes it far easier to discover and exploit. Stale service accounts, excessive privileges and unclear ownership become much more serious once agents can search and act across them at machine speed.
Then set autonomy deliberately. Not every agent needs the same freedom. Base it on the consequence of a mistake, how easily it can be reversed and the authority the task requires.
A good autonomy policy defines the maximum authority, transaction limits, escalation triggers, the separation between preparing an action and executing it, and how quickly autonomy can be reduced if something changes. The anti-pattern that undermines all of this is giving an agent broad permissions and relying on the model to decide when not to use them.
If this agent makes the wrong decision at 2 a.m., what is the maximum consequence it can create before another control stops it?
10. Ten actions to take before agents go into production
Tick the ones already in place in your organisation. Your answers stay in your browser.
The goal is not to remove AI autonomy or put a person behind every action. It is bounded autonomy: enough freedom for AI to create value, with clear limits on where trust ends and control begins.