Wisdom Of The (AI) Crowd: Investigating Artificial Swarm Intelligence In Large Language Models

2026-06-30 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A study investigated whether large language models (LLMs) can replicate human swarm intelligence effects, which typically offer high collective accuracy but face scalability issues. Researchers conducted a controlled experiment using 960 manually executed prompts across GPT-5, Gemini 2.5 Pro, and Claude Sonnet 4.5. The experiment tested both intra-model sampling and inter-model aggregation on eight distinct estimation tasks. Findings consistently showed error reduction through both aggregation methods, with significant reductions up to 37 percentage points in Mean Absolute Percentage Error (MAPE) across various strategies. The research also identified positive correlations (Spearman's ρ=0.242-0.568, all p<0.001) between relative confidence interval widths and estimation errors, suggesting LLMs exhibit metacognitive awareness regarding uncertainty. This work explores deploying LLM swarms in organizational decision-making.

Key takeaway

For Directors of AI/ML evaluating decision support systems, consider implementing LLM swarm intelligence. Your teams can achieve up to 37 percentage points error reduction by aggregating responses from models like GPT-5 or Gemini 2.5 Pro. This approach enhances accuracy and leverages LLMs' inherent metacognitive awareness of uncertainty. Explore intra-model sampling and inter-model aggregation strategies to improve organizational decision-making processes.

Key insights

LLM swarms can reduce estimation errors and exhibit metacognitive awareness, mimicking human swarm intelligence.

Principles

• Aggregation reduces LLM estimation errors.
• LLMs show metacognitive uncertainty awareness.
• Intra- and inter-model aggregation are effective.

Method

A controlled experiment used 960 prompts across GPT-5, Gemini 2.5 Pro, and Claude Sonnet 4.5, testing intra-model sampling and inter-model aggregation on eight estimation tasks.

In practice

• Deploy LLM swarms for decision-making.
• Use aggregation to improve LLM accuracy.
• Explore LLM uncertainty for better estimates.

Topics

• Large Language Models
• Swarm Intelligence
• Model Aggregation
• Metacognition
• Decision Support Systems
• GPT-5

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, Director of AI/ML, Consultant

Related on AIssential

• MetaCogAgent: A Metacognitive Multi-Agent LLM Framework with Self-Aware Task Delegation — Metacognitive multi-agent LLM systems improve task accuracy and efficiency by enabling self-assessment and adaptive delegation.
• Cognitive Effects and Biases in Large Language Models — This tutorial clarifies cognitive effects and biases in LLMs by bridging psychology and NLP.
• IEEE Transactions on Artificial Intelligence, Volume 7, Issue 2, February 2026 — The volume showcases diverse AI advancements, from LLM applications in autonomous driving to specialized models for protein language processing.
• Nous: An Attempt to Extract and Inject the Cognition Behind Prediction-Market Behavior — LLM agent ensembles face cognitive monoculture; prompt-level injection of human behavioral profiles fails to induce diversity.
• A Systematic Evaluation of Black-Box Uncertainty Estimation Methods for Large Language Models — Black-box uncertainty estimation for LLMs lacks a dominant method, but candidate comparison and hybrid approaches show promise.
• Beyond Static Responses: Multi-Agent LLM Systems as a New Paradigm for Social Science Research — A six-tier framework maps LLM agent evolution from static tools to complex adaptive systems for social science research.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.