Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents?

2026-07-08 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, quick

Summary

Hierarchical search agents, which use large language models to decompose complex questions into sub-queries for parallel sub-agents, typically instantiate all roles with a single model of identical scale. This research investigates optimal model capacity distribution by factorizing hierarchical search into delegation, execution, and a fixed answer generation role. Experiments on five multi-hop QA benchmarks reveal that role factorization consistently outperforms single-agent baselines, improving exact match from 4.5 to 8.6 points. Crucially, capacity sensitivity is asymmetric; scaling the delegation backbone yields an ~11-point EM improvement, while scaling the execution sub-agent only provides ~2.6 points, identifying decomposition as the primary capability bottleneck. Furthermore, a 1.7B-parameter executor, trained via quality-filtered trajectory distillation, achieves accuracy comparable to a frontier sub-agent while consuming 37% fewer sub-agent tokens.

Key takeaway

For AI Engineers designing multi-agent systems for complex QA, you should prioritize allocating significant model capacity to the delegation component. This strategy, focusing on task decomposition, yields substantial performance gains, as scaling delegation improves exact match by ~11 points. Conversely, you can downsize your execution sub-agents, potentially using smaller models like a 1.7B-parameter executor, to reduce token consumption by 37% without sacrificing accuracy, optimizing resource allocation.

Key insights

Concentrating model capacity in the delegation role significantly improves hierarchical search agent performance, while execution can be downsized.

Principles

• Role factorization enhances multi-agent system performance.
• Delegation is the primary bottleneck in hierarchical search.
• Asymmetric capacity distribution is optimal for multi-agent systems.

Method

Factorize hierarchical search into delegation, execution, and answer generation roles. Conduct controlled capacity sweeps on delegation and execution axes to identify bottlenecks.

In practice

• Concentrate model capacity on the delegation component.
• Downsize the execution sub-agent without accuracy loss.
• Utilize quality-filtered trajectory distillation for smaller executors.

Topics

• Hierarchical Search Agents
• Multi-agent Systems
• Model Capacity Allocation
• Task Decomposition
• Multi-hop QA
• Trajectory Distillation

Code references

• QinnanCai0115/role-factorized-search

Best for: Research Scientist, AI Architect, NLP Engineer, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• What Are Hierarchical AI Agents? Solving Context & Task Challenges — Hierarchical AI agents mitigate single-agent limitations through structured delegation and specialized roles.
• AI Agents of the Week: Papers You Should Know About — Less context can improve coding agent performance and reduce inference costs.
• Scaling up multi-agent systems: an interview with Minghong Geng — Hierarchical and hybrid designs are key to scaling multi-agent systems for complex, long-horizon problems.
• Google Explores Scaling Principles for Multi-agent Coordination — Multi-agent systems offer conditional benefits, excelling in parallelizable tasks but hindering sequential ones.
• Building Search for AI Agents with Exa CEO Will Bryk — AI agents demand a new search paradigm focused on comprehensive, controllable, and deep information retrieval, distinct from human-optimized engines.
• Why More Context Makes Your Agent Dumber and What to Do About It — Nupur Sharma, Qodo — LLM agents struggle with large contexts, planning loops, and task overload; structured multi-agent systems and context optimization are key.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.