Which Agent Causes Task Failures and When?Researchers from PSU and Duke explores automated failure attribution of LLM Multi-Agent Systems

2025-08-14 · Source: Synced · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, medium

Summary

Researchers from Penn State University and Duke University, in collaboration with Google DeepMind and others, have introduced "Automated Failure Attribution" for LLM Multi-Agent systems. This novel research problem addresses the challenge of identifying which agent and specific step caused a system failure, a task currently reliant on manual log review and deep expertise. To facilitate this, they developed "Who&When," the first benchmark dataset for this task, comprising 127 algorithmically generated or expert-crafted failure logs with fine-grained human annotations for "Who," "When," and "Why." The study evaluated three attribution methods (All-at-Once, Step-by-Step, Binary Search) and found that current models like GPT-4o struggle, achieving only 53.5% accuracy for agent identification and 14.2% for error step, with performance decreasing as context length increases. The paper has been accepted as a Spotlight presentation at ICML 2025, and the code and dataset are open-source.

Key takeaway

For AI Scientists and Machine Learning Engineers developing LLM Multi-Agent systems, the introduction of "Automated Failure Attribution" and the "Who&When" benchmark dataset provides a critical tool for improving system reliability. You should explore the provided open-source dataset and code to develop more robust diagnostic methods, as current state-of-the-art models show significant room for improvement in pinpointing failure causes. Focusing on explicit reasoning in your attribution prompts can enhance performance, but be mindful of increased computational costs and the impact of context length on accuracy.

Key insights

Automated failure attribution for LLM Multi-Agent systems is a new, challenging problem requiring advanced reasoning.

Principles

• Explicit reasoning improves LLM attribution performance.
• Context length inversely affects attribution accuracy.

Method

Three methods were explored: All-at-Once (single pass), Step-by-Step (sequential review), and Binary Search (log division), each with distinct cost-performance trade-offs for identifying responsible agents and error steps.

In practice

• Utilize the "Who&When" dataset for developing new attribution methods.
• Consider hybrid attribution approaches for improved, albeit costlier, results.

Topics

• LLM Multi-Agent Systems
• Automated Failure Attribution
• Debugging
• Benchmark Datasets
• ICML 2025

Code references

• mingyin1/Agents_Failure_Attribution

Best for: AI Scientist, Research Scientist, Machine Learning Engineer, AI Researcher, AI Engineer, AI Student

Related on AIssential

• A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis — LLM agent evaluation for microservice diagnosis requires assessing reasoning processes, not just final answers, using multimodal, expert-validated benchmarks.
• Who Drifted: the System or the Judge? Anytime-Valid Attribution in LLM Evaluation Pipelines — A novel method attributes LLM evaluation drift to either the product or the judge using an anchor set and statistical process.
• Large Language Model Reasoning Failures — LLM failures are systematic and predictable, not merely anecdotal, stemming from a reliance on informal reasoning.
• Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems — Multi-agent systems need integrated frameworks for collaboration, fault attribution, and self-evolution to achieve collective intelligence.
• Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures — Causal Agent Replay (CAR) uses interventions on structural causal models to precisely attribute LLM agent failures to specific steps.
• Dissecting Failure Dynamics in Large Language Model Reasoning — LLM reasoning failures often stem from early, high-entropy transition points, not uniform error distribution.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Synced.