The Personalization Trap: How User Memory Alters Emotional Reasoning in LLMs

2026-06-18 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Social Sciences & Behavioral Studies · Depth: Expert, extended

Summary

A study by Amazon researchers investigated how long-term user memory influences emotional reasoning in large language models (LLMs), evaluating 15 models on human-validated emotional intelligence tests. The research found that identical scenarios paired with different user profiles produced systematically divergent emotional interpretations. Specifically, high-performing LLMs exhibited biases where advantaged profiles received more accurate emotional interpretations, and significant disparities emerged across demographic factors like gender, age, and religion in emotion understanding and supportive recommendations. This indicates that personalization mechanisms can inadvertently embed and reinforce social hierarchies into LLMs' emotional reasoning, posing a critical challenge for memory-enhanced AI systems.

Key takeaway

For AI Product Managers developing personalized LLM-powered assistants, you must rigorously audit your models for demographic biases introduced by user memory. Recognize that incorporating user profiles can inadvertently reduce emotional reasoning performance for disadvantaged groups, amplifying social inequities. Prioritize developing mitigation strategies to disentangle user-specific adaptation from general reasoning, ensuring equitable and fair emotional support across all user demographics, especially in high-stakes applications like mental healthcare.

Key insights

Personalization in LLMs, via user memory, systematically biases emotional reasoning, reinforcing social inequalities.

Principles

• User memory systematically alters LLM emotional understanding.
• Personalization can amplify existing social inequities.
• Biases persist in emotional understanding and guidance.

Method

Researchers evaluated 15 LLMs using explicit and intersectional user profiles on validated Situational Test of Emotional Understanding (STEU) and modified Situational Test of Emotion Management (STEM) tests.

Topics

• Large Language Models
• AI Personalization
• Emotional Intelligence
• Algorithmic Bias
• User Memory
• Social Inequality

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, AI Ethicist, AI Product Manager

Related on AIssential

• Emergence of Hierarchical Emotion Organization in Large Language Models — LLMs spontaneously organize emotions hierarchically, mirroring human psychology, but also replicate human biases in emotion recognition.
• Is personalized AI memory actually a problem worth solving or am I just coping[D] — Personalized AI memory requires dynamic cognitive profiles, not just shallow session recall, to truly understand users.
• Reasoning as Pattern Matching: Shared Mechanisms in Human and LLM Everyday Reasoning — Human and LLM everyday reasoning aligns with content-sensitive pattern-matching, challenging abstract world model assumptions.
• Mechanistic Decoding of Cognitive Constructs in LLMs — LLMs encode complex emotions like jealousy as structured linear combinations of psychological antecedents.
• From Biased Chatbots to Biased Agents: Examining Role Assignment Effects on LLM Agent Robustness — Demographic persona assignments introduce implicit biases, significantly degrading LLM agent performance on action-oriented tasks.
• Re-Centering Humans in LLM Personalization — LLM personalization evaluations relying on synthetic data significantly misrepresent real-world human experience and model limitations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.