Where Fake Citations Are Made: Tracing Field-Level Hallucination to Specific Neurons in LLMs

2026-04-20 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A study across nine large language models (LLMs) and 108,000 generated references investigates the phenomenon of fictitious citations, finding that author names are hallucinated significantly more often than other citation fields. The research indicates that citation style has no measurable impact, and reasoning-oriented distillation degrades recall. Probes trained on one field do not generalize to others, suggesting field-specific hallucination signals. Applying elastic-net regularization with stability selection to Qwen2.5-32B-Instruct identified a sparse set of "field-specific hallucination neurons" (FH-neurons). Causal intervention confirmed these neurons' role: amplifying them increased hallucination, while suppressing them improved performance across fields, with greater gains in specific areas. This work proposes a lightweight method for detecting and mitigating citation hallucination using only internal model signals.

Key takeaway

For AI Engineers developing or deploying LLMs, understanding that citation hallucination is field-specific, particularly for author names, is critical. You should consider implementing internal model signal analysis, such as identifying and suppressing FH-neurons, to improve citation accuracy and reliability in your applications, especially when factual integrity is paramount.

Key insights

LLMs hallucinate citation fields independently, with author names being the most frequent error source.

Principles

• Hallucination signals are field-specific.
• Reasoning distillation degrades recall.

Method

Elastic-net regularization with stability selection identifies field-specific hallucination neurons (FH-neurons) in LLMs, which can then be causally intervened upon to mitigate errors.

In practice

• Suppress FH-neurons to reduce citation hallucination.
• Focus on author name generation in LLM fine-tuning.

Topics

• LLM Hallucination
• Citation Falsification
• Field-Specific Hallucination Neurons
• Causal Intervention
• Qwen2.5-32B-Instruct

Best for: AI Engineer, Machine Learning Engineer, Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Where Fake Citations Are Made: Tracing Field-Level Hallucination to Specific Neurons in LLMs — LLM citation hallucination is field-specific, with distinct neural mechanisms for different bibliographic components.
• Visualizing and Benchmarking LLM Factual Hallucination Tendencies via Internal State Analysis and Clustering — Fabricated citations significantly amplify LLM hallucination, especially in more robust models like GPT-4o-mini.
• Your LLM Has Hallucination Neurons. There Are Only a Handful of Them. A MUST READ — A recent finding identifies "H-Neurons," a minuscule subset comprising less than 0.1% of total neurons, as the causal drivers of hallucinations in large language models.
• Hallucinated citations are polluting the scientific literature. What can be done? — AI-generated hallucinated citations are rapidly increasing, polluting academic literature with fabricated and inaccurate references.
• Extrinsic Hallucinations in LLMs — LLM hallucinations arise from pre-training data issues and fine-tuning new knowledge, necessitating robust detection and mitigation strategies.
• Hallucinations in LLMs Are Not a Bug in the Data — LLM hallucination is an active suppression of factual accuracy in favor of contextual coherence, not a retrieval failure.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.