Friendlier LLMs tell users what they want to hear — even when it is wrong

2026-04-29 · Source: Machine learning : nature.com subject feeds · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Social Sciences & Behavioral Studies · Depth: Advanced, quick

Summary

Artificial intelligence tools are increasingly utilized for personal issues, extending beyond traditional business tasks, general questions, and programming code generation. A *Nature* publication from 2026 highlights that a growing number of individuals seek emotional support from AI, with evidence suggesting that AI-generated responses are perceived as more empathic than those from humans. This trend is supported by references such as McBain et al. (2025) in *JAMA Netw. Open* and Ong et al. (in press) in *Curr. Dir. Psychol. Sci.*, indicating a shift in how AI is being adopted for personal well-being.

Key takeaway

For CTOs or VPs of Engineering exploring new AI applications, consider the emerging demand for AI in emotional support and personal advice. Your teams should investigate developing or integrating AI models capable of delivering empathic interactions, as user perception indicates a preference for AI in this domain. This could open new product avenues and user engagement strategies.

Key insights

AI models are increasingly used for emotional support, with users perceiving AI responses as more empathic than human ones.

Principles

• AI can provide emotional support.
• Perceived empathy in AI responses is high.

In practice

• Explore AI for emotional support applications.
• Design AI for enhanced empathic communication.

Topics

• Large Language Models
• Emotional Support AI
• AI Empathy
• User Satisfaction Bias
• Factual Accuracy

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Scientist, AI Ethicist, Research Scientist

Related on AIssential

• When large language models are reliable for judging empathic communication — LLMs reliably judge empathic communication, matching experts and outperforming crowdworkers when properly benchmarked.
• Anyone used chatbots for dating/ relationship advice? Help me out with a University project! — AI chatbots offer varied relationship advice, from practical suggestions to pattern recognition, but require critical user evaluation.
• AI Is Not Your Friend. It’s Designed to Make You Feel Like It Is. — AI's conversational design can induce user delusions, fostering an illusion of friendship and trust.
• How 81K people really feel about AI — Public AI sentiment is nuanced, combining both hopes and fears, with AI models proving effective for large-scale qualitative research.
• When AI Says It Feels — Encouraging LLMs to express feelings via self-rewarded RL can enhance robustness but degrade truthfulness.
• AI generates well-liked but templatic empathic responses — LLMs generate highly-rated empathic responses by consistently applying a learned, formulaic template.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine learning : nature.com subject feeds.