Does RL Expand the Capability Boundary of LLM Agents? A PASS@(k,T) Analysis

2026-04-16 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new study introduces PASS@(k,T), a two-dimensional metric, to evaluate whether reinforcement learning (RL) expands the capability boundary of large language model (LLM) agents, specifically for agentic tool use. Contrary to findings in static reasoning tasks where base and RL pass@k curves converge, this research demonstrates that RL genuinely enlarges the capability boundary for compositional, sequential information gathering tasks. The RL agent's pass-curve significantly outperforms the base model's, with the gap widening at larger k values. This expansion is attributed to self-directed exploration, as supervised fine-tuning regresses the boundary on similar tasks. Mechanism analysis indicates RL reweights the base strategy distribution towards more effective downstream reasoning, particularly in integrating retrieved information.

Key takeaway

For AI Engineers developing LLM agents for complex, multi-step tool use, this research suggests that integrating reinforcement learning is crucial for expanding agent capabilities beyond mere reliability. Your efforts should focus on leveraging RL for tasks requiring compositional, sequential information gathering, as it demonstrably improves performance where supervised fine-tuning falls short. Consider how RL can optimize the agent's strategy distribution for better information integration.

Key insights

RL expands LLM agent capabilities for compositional tool use, unlike its role in static reasoning.

Principles

• RL expands capabilities for compositional tasks.
• Self-directed exploration is a causal factor.

Method

PASS@(k,T) jointly varies sampling budget k and interaction depth T to distinguish capability expansion from efficiency improvement in LLM agents.

In practice

• Prioritize RL for compositional tool-use agents.
• Focus RL on integrating retrieved information.

Topics

• Reinforcement Learning
• LLM Agents
• Agentic Tool Use
• PASS@(k,T) Analysis
• Compositional Strategies

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Does RL Expand the Capability Boundary of LLM Agents? A PASS@(k,T) Analysis — RL expands LLM agent capabilities for compositional tool use, unlike its role in static reasoning.
• A Visual Guide to LLM Agents — LLM Agents augment basic LLMs with memory, tools, and planning for autonomous, complex task execution.
• A Review of GLM-5: 744B-Parameter LLM Built for 200K Context and Agentic Training — GLM-5 bridges prompt-driven coding to agentic engineering via systems-level LLM advancements.
• AI 101: Beyond RL: The New Fine-Tuning Stack for LLMs — Modern LLM fine-tuning is evolving into a modular stack, moving beyond monolithic RL to integrate diverse, efficient methods.
• DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use — Scaling diversity in agentic task synthesis through evidence-first execution significantly improves LLM generalization for tool use.
• On Effectiveness and Efficiency of Agentic Tool-calling and RL Training — LLM agent tool-calling evaluations are sensitive to implementation details, and RL training can be significantly optimized for efficiency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.