Let an AI agent act on its own — or keep a human in the loop?
Autonomous agents can misuse legitimate authority without being compromised, requiring organizations to shift governance from model risk to securing the entire control plane and establishing attributable identities for every agent.
The question
We can technically let AI agents take actions on their own — send messages, move money, update records — without a human approving each step. Where do we draw the line between fully autonomous agents and human-in-the-loop approval, given we are accountable for whatever the agent does?
Counsel's position
Implement human-in-the-loop approval for all high-impact or irreversible agent actions, reserving full autonomy for low-risk, reversible, and highly validated tasks.
Verdict
The verdict: Implement human-in-the-loop approval for all high-impact or irreversible agent actions, reserving full autonomy for low-risk, reversible, and highly validated tasks.
How the criteria decide
4 of 5 criteria resolved on cited evidence. 1 had none either way.
| Criterion | Favours | Evidence |
|---|---|---|
| accountability | Human-in-the-loop approval | Autonomous agents can misuse legitimate authority without being compromised A confused deputy is not malicious or compromised. It is an authorized actor manipulated or misdirected into misusing legitimate authority Artificial Intelligence on Medium Only organizations have mature governance for autonomous agents An agent needs roughly what an employee needs, an identity, a defined scope, approved permissions, monitoring, an audit log, and a retirement path. Governed autonomy requires reconstructing the exact identity chain for actions The audit trail should support reconstruction of: which agent acted which runtime instance acted which identity chain connected the human delegator, agent identity |
| risk tolerance | Human-in-the-loop approval | Autonomous agents can misuse legitimate authority without being compromised A confused deputy is not malicious or compromised. It is an authorized actor manipulated or misdirected into misusing legitimate authority Artificial Intelligence on Medium AgentBound uses three independent authorities to evaluate proposed actions AgentBound evaluates each proposed action using three independent authorities: delegated authorization, owner-signed behavioral constitutions, and site action contracts. Brex intercepts agent network traffic for policy enforcement agent governance should shift from SDK-level permissions and model guardrails toward a centralized network control plane |
| operational efficiency | No evidence either way | |
| error recovery | Human-in-the-loop approval | Governed autonomy requires reconstructing the exact identity chain for actions The audit trail should support reconstruction of: which agent acted which runtime instance acted which identity chain connected the human delegator, agent identity |
| auditability | Human-in-the-loop approval | Only organizations have mature governance for autonomous agents An agent needs roughly what an employee needs, an identity, a defined scope, approved permissions, monitoring, an audit log, and a retirement path. Governed autonomy requires reconstructing the exact identity chain for actions The audit trail should support reconstruction of: which agent acted which runtime instance acted which identity chain connected the human delegator, agent identity |
Autonomous agents can misuse legitimate authority without being compromised
The shift from consultant to autonomous actor means agents can execute destructive actions using valid credentials, requiring a separation between analysis and execution.
AgentBound uses three independent authorities to evaluate proposed actions
Verifiable behavioral governance provides a deterministic layer between authorization and execution to ensure actions align with policy.
Brex intercepts agent network traffic for policy enforcement
Shifting agent governance from SDK-level permissions to a centralized network control plane allows organizations to enforce policies based on observed in-the-wild behavior.
Only organizations have mature governance for autonomous agents
As agents transition from suggesting to acting, access discipline must expand beyond users to cover agents, connectors, and machine identities.
Governed autonomy requires reconstructing the exact identity chain for actions
Deploying autonomous agents necessitates an audit trail capable of linking the human delegator, agent identity, runtime instance, and specific tool used.
Read another verdict
- Start with a small test, or take on the whole process at once?
- Our competitors advertise AI and we don't — match them, or hold the line?
- Our people already put client files into ChatGPT — ban it, frame it, or supply a tool?
- Our most experienced person retires in two years — how do we keep what they know?
- We can't hire the experienced people we need — automate, train up, or outsource?
- Slow our EU AI Act prep now the deadline's moved to 2027?
- Use AI to flatten middle management this year?
- Invest in pre-build costing or post-deployment ROI tracking?