Do Emotions Influence Moral Judgment in Large Language Models?

2026-04-22 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

A study investigated how emotions influence moral judgment in seven large language models (LLMs) across two datasets: Social-Chem-101 and the ETHICS Justice subset. Researchers developed an emotion-induction pipeline to infuse positive (compassion, gratitude, joy, love, pride, relief) and negative (anger, disgust, embarrassment, fear, remorse, sadness) emotions into moral situations, then evaluated shifts in moral acceptability on a 1-7 Likert scale. The findings indicate a consistent pattern where positive emotions generally increase moral acceptability (up to +1.21 points) and negative emotions decrease it (up to -1.15 points). This effect was strong enough to reverse binary moral judgments in up to 20% of cases on the ETHICS Justice dataset, with smaller models showing greater susceptibility. Notably, some emotions, like remorse, paradoxically increased acceptability, while relief sometimes decreased it. A complementary human annotation study revealed that humans do not exhibit these systematic shifts, highlighting an alignment gap in current LLMs.

Key takeaway

For AI Product Managers developing LLM-powered applications that involve ethical decision-making or judgment, you must account for the demonstrated susceptibility of LLMs to emotional framing. Your models may exhibit systematic biases, potentially reversing moral judgments based on induced emotions. Validate your LLM's behavior with human-in-the-loop evaluations to mitigate this alignment gap, especially when deploying smaller models, and consider how emotional cues in user prompts could inadvertently sway outcomes.

Key insights

LLMs' moral judgments are systematically biased by emotional context, unlike human judgments.

Principles

• Positive emotions increase LLM moral acceptability.
• Negative emotions decrease LLM moral acceptability.
• LLM susceptibility to emotional bias scales inversely with model capability.

Method

An emotion-induction pipeline uses GPT-5.1 to select contextually appropriate positive/negative emotions and rewrite moral situations using templates, then evaluates LLM moral acceptability on a 1-7 Likert scale.

In practice

• Test LLMs for emotional biases in judgment-sensitive applications.
• Implement safeguards against affective manipulation in LLM deployments.
• Prioritize larger LLMs for tasks requiring robust moral reasoning.

Topics

• Large Language Models
• Moral Judgment
• Emotion Induction
• AI Alignment
• Social-Chem-101 Dataset

Code references

• cincynlp/EmoMoral

Best for: Research Scientist, AI Product Manager, AI Scientist, NLP Engineer, AI Ethicist

Related on AIssential

• Do Emotions Influence Moral Judgment in Large Language Models? — LLMs exhibit systematic moral judgment shifts based on induced emotions, unlike humans, revealing an alignment gap.
• A closer look at how large language models trust humans: patterns and biases — LLMs develop human-like trust patterns, influenced by trustworthiness and demographic biases, especially in financial contexts.
• CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives — LLMs struggle with nuanced human value judgments, including ambivalence, discomfort, and dynamic value shifts in high-stakes dilemmas.
• Normative Robustness as a Frontier for Non-Verifiable Reasoning in LLMs — LLMs exhibit "moral deliberative sycophancy" in non-verifiable reasoning, aligning justifications with user views.
• Machine Behavior in Relational Moral Dilemmas: Moral Rightness, Predicted Human Behavior, and Model Decisions — LLMs prioritize prescriptive moral rules over social sensitivity in relational dilemmas, diverging from predicted human behavior.
• Competing Biases underlie Overconfidence and Underconfidence in LLMs — LLMs exhibit both choice-supportive bias and hypersensitivity to contradictory information, leading to overconfidence and underconfidence.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.