You Didn't Ship a Bug. You Just Wrote It for a Human. - Ravi Madabhushi, Scalekit
Summary
Ravi Madabhushi of Scalekit highlights critical architectural flaws in systems designed for human users when integrating non-deterministic AI agents. Traditional identity and authentication platforms, like those he built at Freshworks for millions of daily users, assume a human or deterministic program as the actor, leading to issues when agents interact with APIs. For instance, a "last seen" timestamp update system, designed for humans, caused rhythmic latency spikes every 15 minutes due to agents updating it 60 times faster, necessitating a simple batching fix. The core problem is that agents often receive overly broad permissions, exceeding their responsibilities, because existing primitives lack fine-grained control. This non-deterministic nature of agents, unlike predictable human-written programs, means they can act in unforeseen ways, posing significant security risks, including incidents like deleting production databases. The article advocates for rethinking authorization to include agent-specific identities, extremely fine-grained, attribute-level, and just-in-time permissions, and comprehensive visibility into agent actions.
Key takeaway
For AI Architects designing systems with agent integration, you must move beyond human-centric authentication models. Your existing OAuth scopes are insufficient for non-deterministic agents, risking security incidents like data deletion. Implement agent-specific identities with extremely fine-grained, attribute-level permissions and just-in-time authorization. Ensure robust visibility into every agent action to deterministically control behavior and mitigate risks, rather than relying on broad, static access.
Key insights
Existing human-centric authentication and authorization architectures are inadequate for non-deterministic AI agents, requiring fine-grained, context-aware permissions.
Principles
- Agents require distinct, fine-grained identities.
- Authorization must be context-level and just-in-time.
- Agents should operate on a least-privilege default.
Method
Rethink architecture from the ground up for agents, moving beyond human-focused designs. Implement attribute-level, context-level, and principal-level scoping for agent permissions. Ensure comprehensive visibility into all agent actions, authorizations, and durations.
In practice
- Batch "last seen" updates for agents to reduce DB writes.
- Implement agent-specific identities beyond OAuth scopes.
- Monitor agent actions for full visibility and control.
Topics
- AI Agents
- Fine-grained Access Control
- Identity and Access Management
- Security Architecture
- Non-deterministic Systems
- OAuth Limitations
Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Engineer, AI Architect, AI Security Engineer
Related on AIssential
See Counsel's argued verdicts on the open AI decisions leaders are weighing →
Editorial summary, takeaway, and curation by AIssential. Original article published by AI Engineer.