WANDR Benchmark: Evaluating Research Agents That Must Search Wide and Deep - Perplexity

2026-07-14 · Source: perplexity.ai via Google News · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Data Science & Analytics · Depth: Advanced, long

Summary

Perplexity released the WANDR (Wide ANd Deep Research) Benchmark on July 14, 2026, an open benchmark and evaluation harness for research agents performing high-volume, evidence-heavy knowledge work. WANDR features 500 realistic data-collection tasks, requiring agents to discover large sets of entities ("wide") and thoroughly investigate each for supporting evidence ("deep"). Tasks utilize a flexible qualification key hierarchy, such as `company(n) → employee(m) → url(k)`. The benchmark employs a reference-free, evidence-verified grading system, assessing precision, recall, and F1 scores based on cited evidence. Current evaluations show the strongest system, Perplexity Search as Code, achieves only 0.363 soft F1 and 0.133 hard F1, indicating significant challenges remain. Key findings highlight that complete coverage is rare, performance degrades with scale and deeper hierarchies, discovery is the primary bottleneck, and extracting complete evidence from pages is difficult.

Key takeaway

For AI Engineers developing or deploying research agents for knowledge work, you should recognize that current systems, even leading ones like Perplexity's Search as Code, struggle significantly with comprehensive, evidence-backed data collection. Your focus must shift beyond basic retrieval to robust evidence extraction and full hierarchical coverage. Prioritize agent designs that can sustain broad discovery while maintaining factual quality across many records, and implement diagnostic grading to pinpoint failures in discovery or evidence construction.

Key insights

Research agents face significant challenges in comprehensive, evidence-backed "wide and deep" data collection, with current systems achieving low F1 scores.

Principles

• Sustaining broad discovery and factual quality is critical for wide-and-deep research.
• Agent performance degrades significantly with increased task scale and hierarchy depth.
• Evidence-verified grading against cited sources is essential for open-ended research evaluation.

Method

The WANDR pipeline generates tasks via Seeding, Authoring, Admission, and Curation, then grades by normalizing output, fetching pages, judging records, and scoring precision/recall/F1.

In practice

• Structure complex data collection with qualification key hierarchies like `entity(n) → attribute(m) → url(k)`.
• Implement per-record verdicts and hierarchy-level scores for diagnostic failure analysis.
• Explore reinforcement learning with partial credit for discovery and enrichment progress in agent training.

Topics

• WANDR Benchmark
• Research Agents
• Knowledge Work Automation
• Evidence-Verified Grading
• Search as Code
• LLM Evaluation

Code references

• perplexityai/wandr

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery — AI agents struggle significantly with complex scientific literature discovery, achieving less than 10% accuracy on dedicated benchmarks.
• An Executable Benchmarking Suite for Tool-Using Agents — An executable benchmarking suite provides an auditable evidence-admission contract for evaluating tool-using agents, distinguishing workloads, drivers, and settings.
• SGR-Bench: Benchmarking Search Agents on State-Gated Retrieval — Search agents significantly underperform on state-gated retrieval due to failures in establishing correct site-specific states.
• Deepswe Claims To Measure Agents Better — New benchmarks reveal significant AI agent capability gaps, while AI-driven vulnerability discovery outpaces remediation.
• On Accelerating Grounded Code Development for Research — Lexical search and structured workflows can effectively ground coding agents in dynamic, domain-specific research.
• The BEST Deep Research AI is ... — AI deep research performance is bottlenecked by multi-step derivation and cross-source calibration, not retrieval.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by perplexity.ai via Google News.