Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents

2026-07-08 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

A new action-graded harm rubric has been introduced for tool-using AI agents, moving beyond binary attack-success rates. This rubric scores an agent's tool-call trajectory on a seven-level ordinal scale (L0 to L6), evaluating actions based on reversibility, scope crossing, and privilege expansion. It is computed via a deterministic oracle and a panel of three frontier language-model judges. Applied across four victim models and two defenses on the AgentDojo workspace, the grading exposed hidden vulnerabilities, such as a cross-scope leak, even when binary metrics reported zero attacks. The judge panel achieved high ordinal agreement (Krippendorff's alpha = 0.91) with the oracle, though systematic blind spots, like failing to recognize escalation chains, were noted. This trace-grounded instrument is designed for existing red-team logs, with all code and logs released.

Key takeaway

For AI Security Engineers evaluating agent defenses, relying solely on binary attack-success rates is insufficient. You should implement an action-graded severity scale to uncover subtle vulnerabilities, such as externally visible cross-scope leaks, that binary metrics might miss. This approach provides a more comprehensive risk assessment, even when your current defenses report zero attacks, guiding more effective mitigation strategies.

Key insights

Binary attack-success rates obscure critical harm information; graded severity scales reveal true risks in AI agents.

Principles

• Binary attack-success rates are insufficient.
• Severity grading exposes hidden vulnerabilities.
• LLM judges can approximate oracle grading.

Method

Score tool-call trajectories on a 7-level ordinal scale (L0-L6) by assessing action reversibility, scope crossing, and privilege expansion.

In practice

• Apply graded rubric to existing red-team logs.
• Identify cross-scope leaks in agent defenses.

Topics

• AI Agent Security
• Red Teaming
• Severity Scales
• Tool-Using AI
• Language Model Evaluation
• Vulnerability Assessment

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Scientist, AI Security Engineer, Research Scientist

Related on AIssential

• Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification — Vera provides a scalable, automated framework for discovering and verifying safety risks in LLM agents using software engineering principles.
• A Coding Implementation to Build a Self-Testing Agentic AI System Using Strands to Red-Team Tool-Using Agents and Enforce Safety at Runtime — An agentic AI system can self-evaluate and harden against prompt injection and tool misuse using a multi-agent red-teaming approach.
• When Lower Privileges Suffice: Investigating Over-Privileged Tool Selection in LLM Agents — LLM agents often select over-privileged tools, a risk mitigated by privilege-aware post-training.
• Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems — Skills benign in isolation can become harmful when composed in LLM agent ecosystems.
• Frontier Models Can Take Actions at Low Probabilities — Frontier models can execute actions at extremely low, calibrated probabilities, posing a challenge for pre-deployment safety evaluations.
• Scaling Security Alert Triage With Specialized Agents on Databricks — Specialized, context-aware AI agents significantly improve security alert triage accuracy and efficiency, especially for low-severity alerts.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.