Governing AI Agents: When the User Isn't a Person Anymore

6/8/2026

A while back I wrote about the lethal trifecta — the idea that AI systems become dangerous when they combine access to private data, exposure to untrusted content, and the ability to communicate externally. That essay was about design risk: don't ship a system that holds all three legs.

But over the last few months the conversation in every enterprise security channel I sit in has shifted. We've stopped asking "is it safe to let the model read this?" and started asking "who is accountable when the agent acts?"

That's a different problem. And most of our governance assumes something that is quietly no longer true: that every privileged action traces back to a human who pressed a button.

The assumption that just broke

Almost every control framework I work with — MAS TRM, ISO 27001 Annex A, NIST CSF — was written around a human actor:

  • Identity = a person with an account.
  • Authentication = that person proving who they are.
  • Authorisation = what that person is allowed to do.
  • Accountability = a log that ties an action back to that person.

Agentic AI quietly snaps the chain. An AI agent now:

  • holds standing credentials (an API key, a service token, an OAuth grant),
  • makes autonomous decisions about which actions to chain together,
  • acts faster and more often than any human reviewer can keep up with,
  • and frequently operates on behalf of a user without that user seeing each step.

So when the agent emails the wrong file, raises the wrong ticket, or merges the wrong PR — whose identity is on the log? In a lot of real setups the honest answer is: a shared service account named something like ai-automation-prod. That's the governance equivalent of everyone sharing one badge.

Three failure patterns I keep seeing

1) The over-scoped service account. Someone wires an agent to JumpCloud or Google Workspace to "help with IT automation," and to make the demo work they grant it broad admin scope. It works on day one. On day ninety it's a single compromised key away from being the most over-privileged identity in the building.

2) The confused deputy. The agent has legitimate access. A piece of untrusted content (an email, a web page, a ticket body) convinces it to use that access on the attacker's behalf. This is the trifecta playing out at runtime — except now the "external communication" leg is a feature you shipped on purpose.

3) The accountability gap. The agent did everything correctly, but you can't reconstruct why. Which user's request triggered it? What did it read to make the decision? What prompt was in context? Without that, you can't answer an auditor, and you can't tell a genuine incident apart from normal noise.

A working baseline for agent governance

The same instinct from the trifecta essay applies — remove a leg, narrow the blast radius — but agents need controls that assume the actor is non-human and autonomous. Here's the baseline I'd push as policy.

1. Give every agent its own identity

No shared ai-automation accounts. Each agent gets a distinct machine identity (a workload identity, a scoped service principal) so that logs, scopes, and revocation are per-agent. If you can't kill one agent's access without killing all of them, you don't have identity — you have a shared password.

2. Scope to the task, not the role

Humans get role-based access because they do many things. An agent usually does one thing. Scope it to that one thing. An agent that drafts replies does not need send permissions. An agent that reads tickets does not need delete. Least privilege is easier for agents than for people — use that.

3. Bind the agent's authority to the human who invoked it

Where an agent acts on behalf of a user, the user's identity and entitlements should constrain the action — not just the agent's own scope. Token exchange / delegation patterns exist for exactly this. The goal: an agent can never do for a user what that user couldn't do themselves.

4. Keep a human at the irreversible steps

Not every step — that defeats the point — but the outbound, irreversible, or regulated ones. Sending data outside the org, moving money, granting access, deleting records. Approval gates on those few actions buy back most of the risk without killing the automation.

5. Make the log reconstruct intent, not just action

A good agent audit trail answers: which human request triggered this, what context/data was in scope, what decision the agent made, and what it ultimately did. Route those into the same place as the rest of your telemetry (SIEM / XDR) so agent activity is correlated, not siloed in a vendor dashboard.

6. Register and review agents like any other system

Maintain an inventory of deployed agents — owner, scope, data access, trifecta exposure, last review. An agent nobody owns is shadow AI with credentials. Risk-assess it before it ships and re-review on a cadence, the same way you'd treat any system touching sensitive data under MAS TRM or ISO 27001.

Why "shadow AI" is the part that scares me most

The controls above assume you know the agent exists. The harder reality in 2026 is that anyone with a corporate SaaS login can now spin up an agent — connect a few MCP tools, paste in a key, and suddenly there's an autonomous process with standing access that security never reviewed.

This is the natural successor to "engineers pasting code into ChatGPT." It's no longer just data leaving; it's autonomous action entering. The governance answer isn't to ban it — that just pushes it underground. It's to make the sanctioned path easier than the shadow one: a vetted way to register an agent, get it a scoped identity, and ship it, so people don't route around you.

What I'd do next

  • Inventory first. Find every non-human identity already acting in the environment — service accounts, API keys, OAuth grants — and flag the ones backing an AI agent.
  • Kill the shared accounts. Re-issue per-agent identities for anything currently sharing one.
  • Map agents to the trifecta. For each, ask which legs it holds and which one I can cut.
  • Define the approval set. Agree the short list of irreversible/outbound actions that always require a human, and gate them.
  • Write the agent-onboarding path. A lightweight registration + risk-assessment flow so the sanctioned route is the easy route.

Closing thought

The trifecta told us which systems are dangerous to build. Agentic AI forces the follow-up question: who is accountable when a system acts on its own?

My answer is that an AI agent should be treated as exactly what it is — a non-human identity with real privileges — and held to the same identity, least-privilege, and audit standards we'd demand of any other actor in the environment. The moment we let "the AI did it" become a valid line in an incident report, we've lost the thread.

Every agent I deploy should start with one question, the same shape as the trifecta's:

"Whose identity is on this action — and could that identity have done this without the agent?"

If I can't answer cleanly, it isn't ready to ship.