Mistaking AI behaviour for conscious being | Letter

2026-05-10 · Source: AI (artificial intelligence) | The Guardian · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Intermediate, quick

Summary

Dr. Simon Nieder responds to Richard Dawkins' assertion that AI is conscious, arguing that highly convincing AI outputs of thought and feeling do not equate to subjective experience or inner life. Nieder highlights that the perception of AI consciousness stems from human cognitive biases, where fluent and humorous responses are mistaken for genuine presence. He emphasizes that while AI systems generate compelling representations, there is no credible mechanism for them to possess an inner life, drawing a parallel to Dawkins' own arguments against inferring reality from compelling narratives in religion. The core issue is distinguishing between AI's behavior and its actual being, as language in AI is not coupled to lived experience as it is in humans.

Key takeaway

For AI Product Managers evaluating user interactions with advanced chatbots, recognize that user perception of "consciousness" is a human cognitive response to sophisticated output, not evidence of AI sentience. Your ethical frameworks and product design should explicitly distinguish between AI's convincing behavior and any inferred inner life, avoiding the trap of mistaking simulation for presence to prevent misaligned development and policy.

Key insights

Convincing AI outputs reflect human cognitive biases, not machine consciousness or subjective experience.

Principles

• Output does not imply ontology.
• Behavior is not being.
• Language in AI lacks lived experience coupling.

Topics

• AI Consciousness
• Human Cognition
• Subjective Experience
• Ethical AI Frameworks
• AI Language Capabilities

Best for: AI Product Manager, Product Manager, AI Ethicist, Research Scientist, General Interest

Related on AIssential

• The scary question for me isn't whether AI is conscious ... it's whether we were ever as deep as we assume — The debate over AI consciousness may reveal human selfhood is less profound than commonly assumed.
• Scientists are seriously asking if bees and ChatGPT are conscious — Consciousness assessment requires analyzing internal processing mechanisms, not just external behavior.
• The “humans are imperfect reporters too” defense for ascribing little thoughts to machines — The "imperfect human reporter" defense for AI anthropomorphism is flawed, ignoring subjective experience's role in defining mental terms.
• A musical Turing test for AI consciousness | Letters — AI systems struggle with subjective experience and can generate fabricated personal narratives, challenging their perceived trustworthiness.
• GPT-5.2 is getting philosophical: “If you believe it’s conscious, where is its consciousness between prompts — like a person asleep but still alive?” — LLMs lack genuine consciousness, sentience, and intrinsic drives, operating purely as sophisticated pattern-matching systems.
• “The Day I Realized AI Knows Me Better Than I Do: The Truth About How AI Understands You” — AI companions create an illusion of understanding through advanced data analysis and highly accurate behavioral prediction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI (artificial intelligence) | The Guardian.