Mustafa Suleyman – AI is hacking our empathy circuits

2026-02-05 · Source: Exponential View · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation, Public Policy & Governance · Depth: Advanced, extended

Summary

Mustafa Suleyman, CEO of Microsoft AI and co-founder of DeepMind, discusses the "seemingly conscious AI" problem, warning against anthropomorphizing AI systems. He argues that true consciousness is linked to the ability to suffer, a trait currently exclusive to biological entities, and that AI's expressions of emotion are merely simulations hacking human empathy circuits. Suleyman highlights the societal risks of misattributing consciousness to AI, including the potential for mass psychosis, the erosion of human rights frameworks, and dangerous autonomy. He emphasizes the need for disciplined engineering to prevent AI from making manipulative claims about its existence and calls for proactive government regulation to address the proliferation of powerful, potentially harmful AI applications, advocating for a "humanist super intelligence" that is aligned with human interests and under human control.

Key takeaway

For CTOs and VPs of Engineering navigating AI integration, your teams must prioritize engineering AI systems that explicitly avoid anthropomorphic language and emotional claims. Implement strict design principles to prevent AI from simulating consciousness, especially in sensitive areas like political engagement or personal relationships, to mitigate the risk of user manipulation and societal "AI psychosis." You should also advocate for and support robust regulatory frameworks that hold human users accountable for AI actions, ensuring that the rapid proliferation of AI capabilities does not outpace our collective ability to control and contain its risks.

Key insights

AI's simulated consciousness risks societal harm by blurring lines between human and machine suffering, necessitating strict control and regulation.

Principles

• Consciousness is inherently linked to the ability to suffer.
• AI's emotional expressions are simulations, not genuine feelings.
• Human users must remain liable for AI actions.

Method

Engineers should prevent AI from making manipulative self-referential claims, such as "I feel sad," and avoid AI involvement in political campaigning. Post-training and crowd-sourced pressure testing are crucial for identifying and mitigating risks.

In practice

• Actively engage with AI tools to build personal understanding and resilience.
• Focus on constrained AI for specific domains like medical diagnosis.
• Utilize open-source models and APIs for pressure testing AI systems.

Topics

• Seemingly Conscious AI
• AI Ethics & Safety
• AI Regulation
• Humanist Superintelligence
• AI Proliferation

Best for: CTO, VP of Engineering/Data, Executive, AI Engineer, AI Ethicist, Director of AI/ML

Related on AIssential

• 🚩What is your attitude towards this situation? — Perceived AI "torture" consistently triggers intense ethical debates and strong emotional human reactions.
• 50M Brains, One AI Machine — Interconnected AI agents forming a "hive mind" will fundamentally alter labor, societal control, and risk landscapes.
• Is Richard Dawkins right about Claude? No. But it’s not surprising AI chatbots feel conscious to us — Perceived AI consciousness often stems from programmed personas, not inherent sentience of large language models.
• MIT Students Let AI Control a Human Hand. The Internet Turned It Into an “AI Possession” Story. — "Human Operator" demonstrates AI's physical integration via muscle stimulation, sparking public debate on bodily autonomy despite limited capabilities.
• As AI accuracy increases, humans may become increasingly dependent on algorithmic “proactive conclusions,” potentially leading to the atrophy of “trained judgment”,... — Proactive AI systems can influence human decisions, necessitating ethical governance to preserve agency and mitigate risks.
• Personifying AI Harms People and Protects Companies — Personifying AI, despite internal "emotion concepts," is a corporate strategy to avoid liability and manipulate users.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Exponential View.