The Same Email, Signed Differently: Testing Negotiation Bias and Recommendation Stability in LLMs

2026-05-20 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

A paper titled "The Same Email, Signed Differently: Testing Negotiation Bias and Recommendation Stability in LLMs" by Jasmin Heierli and Alexandre de Spindler, presented at the 11th Edition of the Swiss Text Analytics Conference in June 2026, investigates how large language models (LLMs) respond to negotiation scenarios when the sender's identity is manipulated. The research, detailed on pages 191–195 of the proceedings published by the Association for Computational Linguistics, specifically examines potential biases in LLM-generated recommendations and the consistency of these recommendations. It explores whether varying signatures on identical emails influence an LLM's output, thereby assessing its stability and fairness in simulated negotiation contexts. This work highlights critical considerations for deploying LLMs in sensitive communication roles.

Key takeaway

For NLP engineers developing LLM-powered negotiation or communication tools, you should rigorously test your models for implicit biases related to sender identity. Ensure your LLMs provide stable and unbiased recommendations, regardless of superficial input variations like email signatures. This vigilance helps prevent unintended discriminatory outputs and maintains user trust in automated systems.

Key insights

LLMs may exhibit negotiation bias and unstable recommendations based on sender identity cues.

Method

The study tests LLM responses to identical negotiation emails with varied sender signatures to evaluate bias and recommendation consistency.

Topics

• Large Language Models
• Negotiation Bias
• Recommendation Stability
• NLP Ethics
• Text Analytics
• SwissText Conference

Best for: Research Scientist, AI Scientist, NLP Engineer, AI Ethicist

Related on AIssential

• Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts — LLMs exhibit inconsistent biases, preferring human experts in stated trust but algorithms in revealed preference tasks.
• Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curation — LLM-based content curation consistently amplifies polarization and exhibits demographic biases, often resistant to prompt engineering.
• Competing Biases underlie Overconfidence and Underconfidence in LLMs — LLMs exhibit both choice-supportive bias and hypersensitivity to contradictory information, leading to overconfidence and underconfidence.
• Investigating Counterfactual Unfairness in LLMs towards Identities through Humor — LLMs exhibit counterfactual unfairness, applying stricter safety policies and attributing more malicious intent based on speaker-target identity.
• AI chatbots can effectively sway voters – in either direction — AI chatbots effectively sway political opinions by generating numerous factual claims, even if some are inaccurate.
• Extending the Contact Hypothesis: Cross-Linguistic Evaluation of Religion and Nationality Bias When Prompting LLMs in German and Icelandic — LLMs exhibit religion and nationality biases, requiring cross-linguistic evaluation for fairness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.