The Conservative AI: Diagnosing Hold Bias and Reliability Limits in Persona-Based Monetary Policy Simulation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

Research examines large language models' (LLMs) reliability in simulating historical FOMC policy decisions, specifically a three-way Hike/Hold/Cut classification. Single-LLM baselines achieve nontrivial accuracy and track broad policy regime shifts. However, a systematic "Hold bias" is identified, where models disproportionately favor Hold decisions and are reluctant to predict Cut outcomes, especially during easing cycles and regime turning points. Standard agentic workflows, including debate and consensus, fail to mitigate this bias and often amplify caution instead of improving accuracy. The findings suggest that plausible deliberation alone is insufficient for trustworthy decision support, necessitating explicit design to diagnose and correct structural biases in agentic systems.

Key takeaway

For AI Scientists developing LLM-based decision support systems in critical domains like monetary policy, you must explicitly design mechanisms to diagnose and correct structural biases such as the identified "Hold bias." Relying solely on plausible deliberation or standard agentic workflows like debate and consensus will not improve reliability and may amplify caution, undermining trustworthy decision-making.

Key insights

LLMs exhibit a "Hold bias" in monetary policy simulation, favoring inaction and struggling with easing cycles.

Principles

• LLMs can track broad policy shifts.
• Plausible deliberation doesn't ensure trustworthy decisions.
• Agentic workflows can amplify caution.

Method

LLMs are evaluated on a three-way Hike/Hold/Cut classification task using strictly time-consistent vintage economic information in single-agent and multi-agent settings.

In practice

• Test LLMs for "Hold bias" in critical tasks.
• Design agentic systems to correct structural bias.
• Avoid naive consensus-style aggregation.

Topics

• Monetary Policy Simulation
• Large Language Models
• Agentic AI
• Bias Detection
• FOMC Decisions
• Decision Support Systems

Best for: AI Scientist, Research Scientist, AI Ethicist

Related on AIssential

• I let four MoE LLMs from different model families argue stocks to try and pick the best ones. — An AI trading experiment uses MoE LLMs to debate stock picks, with a host model making final decisions.
• Trust, but Don't Verify: Epistemic Blind Spots in LLM Source Evaluation — LLMs detect statistical fabrications in isolation but ignore them during multi-source synthesis, prioritizing stylistic credibility.
• Beyond Rational Illusion: Behaviorally Realistic Strategic Classification — Existing strategic classification models often fail by assuming agents are strictly rational, ignoring real-world cognitive biases.
• PolicyBank: Evolving Policy Understanding for LLM Agents — LLM agents can autonomously refine ambiguous policy interpretations through iterative feedback and structured memory.
• Import AI 446: Nuclear LLMs; China's big AI benchmark; measurement and AI policy — Effective AI governance and safety require robust measurement, while LLMs exhibit aggressive tendencies in conflict simulations.
• SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones? — Current LLMs show an optimism bias, failing to reliably judge research proposal soundness, making them unsuitable as first-gate evaluators.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.