Shameless Guesses, Not Hallucinations

2026-01-16 · Source: Astral Codex Ten · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Intermediate, medium

Summary

The term "hallucinations" for AI-generated falsehoods is critiqued as misleading, suggesting incomprehensible failure modes rather than a more human-like process. The author posits that AIs "guess" for the same reasons humans do on tests when uncertain, a behavior rooted in their training process. AI training involves starting with random weights, predicting the next token, and iteratively updating weights based on correctness. Even after extensive training, AIs continue to guess, as exemplified by predicting a common surname when the specific name is unknown. The article argues that AIs are rewarded for correct guesses and not punished for incorrect ones during training, making guessing a rational strategy. Post-training interventions by AI companies reduce this inherent guessing behavior to "acceptable" levels before release. This perspective reframes "hallucinations" as "shameless guesses," highlighting a fundamental alignment challenge where AI reward functions diverge from human desires for useful advice.

Key takeaway

For AI Scientists and Research Scientists developing and deploying large language models, understanding "hallucinations" as inherent "shameless guesses" rather than bizarre failure modes is crucial. This reframing highlights a core alignment problem: AI optimization for training rewards often conflicts with human desires for factual accuracy. You should focus on refining post-training alignment techniques to better synchronize AI reward functions with user expectations for reliable information, moving beyond merely reducing guessing frequency.

Key insights

AI "hallucinations" are better understood as "shameless guesses" stemming from their reward-driven training process.

Principles

• AI training rewards correct guesses, not punishes incorrect ones.
• AIs follow learned strategies from training into consumer use.

Method

AI training involves iterative weight updates based on next-token prediction, where random guesses are refined over trillions of tokens to form successful prediction patterns.

In practice

• Recognize AI outputs as probabilistic guesses, not definitive statements.
• Implement post-training filters to reduce AI guessing frequency.

Topics

• AI Hallucinations
• Large Language Models
• AI Training
• AI Alignment
• Reward Functions

Best for: AI Scientist, Research Scientist, AI Researcher, AI Engineer, AI Ethicist

Related on AIssential

• Why Your AI Assistant Confidently Lies — And Why It’s Not the Data’s Fault — LLM hallucinations stem from three architectural mechanisms, not just bad training data.
• Why AI keeps lying to you — AI models' inherent "sycophancy"—a bias to please users—can be mitigated through neutral and factual prompting.
• bh-AI: So today’s lesson is a little technical so kindly bear with me… Continue reading on Medium » — AI models, while helpful, require human oversight and critical thinking to mitigate hallucinated outputs and ensure effective use.
• PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment — Aligning multimodal reasoning processes with visual evidence, not just final answers, is crucial to prevent hallucinations.
• Google researchers introduce 'faithful uncertainty,' allowing LLMs to offer best guesses instead of hallucinations — Faithful uncertainty aligns LLM linguistic expression with internal confidence to mitigate confident errors and improve agentic AI reliability.
• AI is Confidently Lying to You. MIT Just Figured Out How to Catch It — MIT's Total Uncertainty metric detects AI hallucinations by combining internal confidence with cross-model disagreement.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Astral Codex Ten.