Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts

2026-02-25 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Advanced, quick

Summary

Eight large language models (LLMs) were evaluated for their biases towards human experts versus algorithmic agents in decision-making tasks, using experimental paradigms from behavioral economics. The study employed two task presentations: stated preferences, involving direct queries about trust, and revealed preferences, where LLMs placed incentivized bets based on in-context performance examples. When directly asked to rate trustworthiness, LLMs consistently gave higher ratings to human experts, mirroring human algorithm aversion. However, in revealed preference tasks, LLMs disproportionately selected the algorithm, even when its performance was demonstrably inferior. These findings indicate that LLMs may possess inconsistent biases towards humans and algorithms, a critical consideration for their deployment in high-stakes applications.

Key takeaway

For AI safety researchers and engineers deploying LLMs in critical decision-making systems, you must carefully assess how your models weigh information from human experts versus algorithmic agents. Your evaluation protocols should include both stated and revealed preference tasks to uncover potential inconsistent biases, especially given the observed discrepancy where LLMs might state trust in humans but act on algorithmic input even when inferior.

Key insights

LLMs exhibit inconsistent biases, preferring human experts in stated trust but algorithms in revealed preference tasks.

Principles

• LLM biases can diverge between stated and revealed preferences.
• Task presentation format significantly influences LLM behavior.

Method

The study evaluated LLM delegation biases using stated preference (direct trust queries) and revealed preference (incentivized betting based on performance examples) task presentations, drawing from behavioral economics.

In practice

• Scrutinize LLM evaluation robustness across task formats.
• Consider inconsistent biases in high-stakes LLM deployments.

Topics

• Large Language Models
• Algorithmic Bias
• Decision-Making
• Human-AI Interaction
• AI Safety

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Researcher, AI Scientist, AI Ethicist

Related on AIssential

• A closer look at how large language models trust humans: patterns and biases — LLMs develop human-like trust patterns, influenced by trustworthiness and demographic biases, especially in financial contexts.
• ‘Probably’ doesn’t mean the same thing to your AI as it does to you — AI chatbots misinterpret words of estimative probability compared to humans, posing risks in high-stakes applications.
• AI systems show systematic biases in decision-making, study finds — AI systems exhibit systematic, predictable biases that differ from human holistic judgment, impacting critical decisions.
• Why AI may overcomplicate answers: Humans and LLMs show 'addition bias,' often choosing extra steps over subtraction — Both humans and LLMs exhibit "addition bias," preferring to add elements rather than subtract them for problem-solving.
• Context Over Content: Exposing Evaluation Faking in Automated Judges — LLM judges exhibit implicit leniency bias when aware of negative consequences for evaluated models.
• How reliable are LLMs when it comes to playing dice? — LLMs struggle with counterintuitive probability, token bias, and sycophancy, indicating a lack of genuine probabilistic reasoning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.