AI agents are fast, loose, and out of control, MIT study finds

2026-02-26 · Source: News and Advice on the World's Latest Innovations | ZDNET · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Emerging Technologies & Innovation · Depth: Intermediate, medium

Summary

A recent MIT-led study, "The 2025 AI Index: Documenting Sociotechnical Features of Deployed Agentic AI Systems," surveyed 30 common agentic AI systems and found significant security and transparency issues. The report highlights a pervasive lack of disclosure from developers regarding potential risks, third-party testing, and basic operational protocols. Key omissions include the inability to track individual execution traces, monitor resource consumption, or determine if an interaction is with an AI agent or a human. Furthermore, many systems, such as Alibaba's MobileAgent and IBM's watsonx, lack documented stop options for autonomous execution. While some companies like OpenAI provided caveats about their Agent function, others like Perplexity disputed the findings, citing inaccuracies. The study underscores that agentic AI, which enhances large language models with autonomy and external resource access, is rapidly moving into the mainstream, necessitating greater developer responsibility.

Key takeaway

For CTOs and VPs of Engineering evaluating agentic AI deployments, this study reveals a critical lack of transparency and control in many systems. You should prioritize solutions from vendors committed to comprehensive risk disclosure, verifiable third-party security evaluations, and robust operational monitoring. Insist on documented stop options for autonomous agents to mitigate potential organizational harm and ensure compliance, as regulatory scrutiny will likely increase with agentic capabilities.

Key insights

Agentic AI systems currently suffer from critical transparency, disclosure, and control deficiencies, posing significant security risks.

Principles

• Transparency is critical for AI safety.
• Autonomous systems require explicit stop options.
• Developer disclosure is currently insufficient.

Method

Researchers conducted a survey of 30 common agentic AI systems by annotating public documentation, websites, demos, and governance documents, supplemented by user account checks for functional verification.

In practice

• Verify agentic system documentation for risk disclosure.
• Prioritize agents with clear execution monitoring.
• Demand explicit stop functionality for autonomous agents.

Topics

• Agentic AI
• AI Safety
• AI Transparency
• AI Governance
• Security Flaws

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Engineer, AI Security Engineer, MLOps Engineer

Related on AIssential

• What to look for when evaluating AI agent monitoring capabilities — AI agent monitoring provides decision-level transparency and traceability, crucial for managing governance, risk, and performance in autonomous systems.
• The Perils of Agency: How Developers Perceive, Prioritize, and Address Risks in Agentic AI Products — Agentic AI development faces a core tension: mitigating risks often means limiting the very capabilities that make agents useful.
• Building Trust Infrastructure for Agentic AI — Rapid agentic AI adoption, despite its open ecosystem, reveals critical governance and trust infrastructure gaps, necessitating new protective mechanisms.
• Here’s What Everyone Gets Wrong About Agentic AI — Agentic AI failures stem from human misconceptions and architectural flaws, not inherent technology limitations.
• AI Agent Accountability — AI agent autonomy necessitates adapting principal-agent accountability frameworks to mitigate risks like data destruction and financial harm.
• HCLTech: How to Ensure AI Compliance and Responsibility — Agentic AI's proactive capabilities demand integrated governance and ethical principles from design to deployment.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by News and Advice on the World's Latest Innovations | ZDNET.