The Pope appears to understand AI better than Geoffrey Hinton does.

2025-06-07 · Source: Marcus on AI · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Intermediate, quick

Summary

This analysis critiques Geoffrey Hinton's recent statements on AI, arguing he misunderstands Large Language Models (LLMs) by conflating their outputs with genuine internal states or consciousness. The author contends that Hinton, like Richard Dawkins previously, fails to distinguish between LLM mimicry and true understanding, which stems from experience. LLMs are characterized as systems that statistically approximate human language by memorizing vast internet data, rather than building mental models. In contrast, Pope Leo XIV's perspective, articulated in a tweet, is highlighted as more accurate: "True comprehension comes from experience, not text approximation." This view aligns with research by Valerio Capraro and Walter Quattrociochi, published in Nature, which asserts LLMs imitate but do not understand, reinforcing that LLMs are "interactive fiction" and not sentient beings.

Key takeaway

For AI ethicists and developers evaluating advanced language models, you must critically assess the underlying mechanisms rather than solely relying on output performance. Recognize that LLMs are sophisticated statistical predictors, not conscious entities, and avoid anthropomorphizing their capabilities. This distinction is crucial for developing responsible AI policies and preventing misinterpretations of AI's true nature and limitations.

Key insights

LLMs mimic language statistically; they do not possess genuine understanding or consciousness derived from experience.

Principles

• Output similarity does not imply similar underlying mechanisms.
• True comprehension requires experience, not just text approximation.
• LLMs are language predictors, not conscious beings.

In practice

• Distinguish LLM mimicry from genuine internal states.
• Evaluate AI based on underlying mechanisms, not just outputs.
• Recognize LLMs as interactive fiction, not sentient entities.

Topics

• Large Language Models
• AI Consciousness
• AI Ethics
• Machine Mimicry
• AI Evaluation
• Cognitive Science of AI

Best for: AI Ethicist, Tech Journalist, General Interest

Related on AIssential

• GPT-5.2 is getting philosophical: “If you believe it’s conscious, where is its consciousness between prompts — like a person asleep but still alive?” — LLMs lack genuine consciousness, sentience, and intrinsic drives, operating purely as sophisticated pattern-matching systems.
• “As our daily lives involve ever more sophisticated computers, we will find that ascribing little thoughts to machines will be increasingly useful in understanding how to get the most good out of them” but “we must be careful not to ascribe properties to a machine that the particular machine doesn’t have” — Imprecise language in AI research risks anthropomorphizing models, leading to misinterpretation and potentially harmful projections.
• Benchmarking LLMs at the Game Of Science (Eleusis) — LLM scientific reasoning involves distinct inductive ability and metacognition, measurable through iterative hypothesis testing.
• The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflows — The LLM fallacy describes users misattributing AI-assisted output to their own competence, distorting self-perception.
• The Fed Chair Just Said What AI Leaders Won't: The Models Don't Work — LLMs are not sufficient for complex system prediction, prescription, and diagnosis due to inherent architectural limitations.
• SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones? — Current LLMs show an optimism bias, failing to reliably judge research proposal soundness, making them unsuitable as first-gate evaluators.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Marcus on AI.