Directional Hallucinations: Ideological Drift in News-Grounded LLM Question Answering

2026-07-25 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

The study "Directional Hallucinations: Ideological Drift in News-Grounded LLM Question Answering" introduces a reproducible framework to measure ideological drift in large language model (LLM) hallucinations. Researchers used 21,727 expert-labeled U.S. political news articles from the QBias corpus, spanning left, center, and right sources. They evaluated three open-weight LLMs (Llama 3 8B Instruct, Mistral 7B v0.3 Instruct, Deepseek 7B Chat) and one proprietary model (GPT-4o-Mini) on document-grounded question answering. Findings indicate hallucination rates vary significantly across models, with Deepseek exhibiting the highest (21.3%) and GPT-4o-Mini the lowest (7.8%). Crucially, hallucinated content consistently showed a robust leftward drift, with 64.2% to 69.8% of hallucinations classified as left-leaning, even when generated from right-leaning sources. Logit-level analysis revealed hallucinations arise in high-entropy contexts, and for Deepseek and Llama, uncertainty predicts leftward drift.

Key takeaway

For MLOps engineers deploying LLMs in civic or election-adjacent information systems, you must implement robust auditing for "directional hallucinations." Your systems should not only aim to reduce overall hallucination rates but also specifically monitor for ideological drift, especially on contested topics like elections or gun control. Consider integrating uncertainty-linked risk indicators to trigger content warnings or abstention policies, as leftward bias persists even when models hallucinate from right-leaning sources.

Key insights

LLM hallucinations in political QA exhibit systematic leftward ideological drift, even from right-leaning sources, often linked to generation uncertainty.

Principles

• Hallucinations are bias-revealing events.
• Uncertainty can trigger directional drift.
• Bias mechanisms vary across models.

Method

A reproducible pipeline for auditing ideological directionality of hallucinated content in news-grounded QA, involving question generation, grounded answering, sentence-level hallucination detection, and ideology classification.

In practice

• Audit LLM deployments for civic information.
• Implement abstention policies for high-uncertainty outputs.
• Target bias mitigation on contested topics.

Topics

• LLM Hallucinations
• Political Bias
• News-Grounded QA
• Ideological Drift
• Model Uncertainty
• AI Auditing

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, MLOps Engineer, AI Ethicist

Related on AIssential

• THE SHAPE OF A MACHINE’S MIND — An LLM's political position is a context-conditioned "shape" in ideological space, not a static "point."
• Can AI Debias the News? LLM Interventions Improve Cross-Partisan Receptivity but LLMs Overestimate Their Own Effectiveness — LLM-based news debiasing improves cross-partisan receptivity when reframing ideology, but LLMs overestimate their own effectiveness.
• MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions — LLM anti-Muslim bias amplifies in complex, real-world deployment conditions, challenging current evaluation and mitigation methods.
• Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologies — LLM-generated encyclopedias like Grokipedia are not inherently neutral, often embedding new ideological biases.
• Polar: A Benchmark for Evaluating Political Bias in LLMs — LLM political bias is context-dependent, varying by language, ideology, and geography.
• The Geometry of Truth: Detecting LLM Hallucinations with Geometric Algebra — Type II hallucinations are context contradictions, dangerous due to their plausibility and standard similarity metric blindness.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.