AI systems show systematic biases in decision-making, study finds

2026-04-20 · Source: Dataconomy · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, quick

Summary

A Hebrew University study, published in the Proceedings of the Royal Society, reveals that AI systems exhibit systematic and predictable biases when evaluating people, differing significantly from human judgment. Analyzing over 43,000 simulated decisions and 1,000 human participants across five scenarios, including financial lending and social assessments, the research found that while both humans and AI favor competent, honest, and well-intentioned individuals, AI uses rigid, segmented criteria rather than holistic impressions. AI biases were often more systematic and sometimes stronger than human biases, favoring older individuals and influenced by religion and gender in financial contexts. The study also highlighted that different AI models produce varied judgments for the same individual, underscoring the critical role of model selection.

Key takeaway

For CTOs and VPs of Engineering deploying AI for critical decision-making like job screening or credit assessment, you must understand that AI's systematic biases are less nuanced and potentially stronger than human biases. Your teams should rigorously audit AI models for specific biases, such as age or gender, and acknowledge that different models yield varied judgments. Prioritize understanding how AI "trusts" individuals rather than merely whether you trust the machine.

Key insights

AI systems exhibit systematic, predictable biases that differ from human holistic judgment, impacting critical decisions.

Principles

• AI evaluates traits segmentally, not holistically.
• AI biases can be more systematic than human biases.
• AI model selection significantly impacts outcomes.

Method

The study analyzed 43,000+ simulated AI decisions and 1,000 human participant evaluations across five financial and social scenarios to compare judgment patterns.

In practice

• Scrutinize AI models for age, religion, and gender biases.
• Compare different AI models for judgment variability.
• Do not assume AI sees people as humans do.

Topics

• AI Systems
• Systematic Biases
• Decision-Making Algorithms
• Human-AI Evaluation Differences
• Algorithmic Bias

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, AI Ethicist, Director of AI/ML

Related on AIssential

• Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts — LLMs exhibit inconsistent biases, preferring human experts in stated trust but algorithms in revealed preference tasks.
• AI is exacerbating gender imbalances in tech and finance — AI is worsening gender disparities in tech and finance through job displacement, adoption gaps, and embedded biases.
• Bias in the machine (edited) — Bias in AI systems stems from historical human prejudices embedded in training data and model design, leading to real-world harm.
• AI as a Double-Edged Sword: Opportunities and Ethical Challenges — AI's dual nature demands responsible development, balancing innovation with ethical safeguards like transparency and human oversight.
• Charting the AI Perception Gap: Across 71 scenarios, AI experts (N=119) and the public (N=1100) have differing views on the risks, benefits, and value of AI. More importantly, AI experts discount the influence of risks stronger than the public does when forming their value judgments [R] — AI experts and the public hold systematically different mental models regarding AI's risks, benefits, and overall value.
• Motivated reasoning, confirmation bias, and AI risk theory — Confirmation bias, driven by motivated reasoning, significantly distorts beliefs in complex fields like AI risk by compounding across cognitive stages.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Dataconomy.