Diverse Evidence, Better Forecasts: Multi-Agent Deliberation Under Information Asymmetry

2026-07-02 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Multi-agent systems are increasingly used for forecasting future events, with deliberation among multiple LLMs believed to improve reasoning and calibration. However, existing approaches often overlook the critical design choice of information distribution, leading to "herding" when all agents receive identical evidence. This research, published on 2026-07-02, proposes "designed information asymmetry" to address this gap by partitioning evidence into shared public and disjoint private subsets. This ensures each agent holds exclusive knowledge, which theoretically reduces inter-agent error correlation. The InfoDelphi framework instantiates this by combining relevance-aware evidence routing, rationale-based iterative deliberation, and confidence-weighted aggregation. On PolyGym, a benchmark of 375 binary forecasting questions from real-world prediction markets, InfoDelphi outperforms single-agent and multi-agent baselines by 12--18% in Brier score and 4--8 percentage points in accuracy, confirming input diversity as key to effective multi-agent reasoning.

Key takeaway

For AI Scientists and Machine Learning Engineers designing multi-agent forecasting systems, you should actively implement designed information asymmetry. Providing each agent with unique private evidence, alongside shared public data, prevents herding and significantly improves forecast accuracy. This approach, exemplified by InfoDelphi, yields 12-18% better Brier scores and 4-8 percentage points higher accuracy, making it essential for robust multi-agent reasoning.

Key insights

Designed information asymmetry, where agents hold unique evidence, is crucial for effective multi-agent deliberation and superior forecasting.

Principles

• Identical evidence leads to multi-agent herding.
• Information asymmetry reduces inter-agent error correlation.
• Input diversity enables effective multi-agent reasoning.

Method

InfoDelphi partitions evidence into public and private subsets, routes evidence by relevance, uses rationale-based iterative deliberation, and aggregates forecasts via confidence-weighting.

In practice

• Partition evidence for multi-agent forecasting.
• Implement relevance-aware evidence routing.
• Use confidence-weighted aggregation for forecasts.

Topics

• Multi-agent systems
• Forecasting
• Information asymmetry
• LLM deliberation
• Brier score
• PolyGym benchmark

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• Distributed Partial Information Puzzles: Examining Common Ground Construction Under Epistemic Asymmetry — Epistemic asymmetry in collaborative tasks challenges AI systems' common ground construction.
• InfoChess: A Game of Adversarial Inference and a Laboratory for Quantifiable Information Control — InfoChess isolates competitive information acquisition as the sole objective in a symmetric adversarial game.
• When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines — Selector quality, not just team diversity, dictates multi-agent LLM pipeline performance.
• More information leads to more accurate beliefs—except in echo chambers — Echo chambers, even with free information flow, can reduce group belief accuracy by reinforcing existing, potentially inaccurate, views.
• Belief Engine: Configurable and Inspectable Stance Dynamics in Multi-Agent LLM Deliberation — The Belief Engine provides auditable, parameter-controlled stance dynamics for LLM agents in deliberative simulations.
• Why a 12-year-old forecasting paper has stood the test of time — Effective forecasting of complex events relies on robust evaluation and intelligent fusion of diverse data sources.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.