FairCoder: Probing LLM Bias in High-Stakes Decision Making via Coding Tasks

2026-07-15 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, extended

Summary

The study introduces "FairCode", a novel benchmark designed to evaluate social bias in large language models (LLMs) for code generation. It comprises two tasks: function implementation and test case generation, covering diverse scenarios like job hiring (540 occupations), college admissions (320 majors), and medical treatment (95 treatments). The benchmark utilizes 955 prompts for function implementation and 72 for test case generation. A new metric, "FairScore", is proposed to quantify bias by combining refusal rate and preference entropy. Experiments on 11 models, including Llama, Mistral, Qwen, and GPT families, reveal that all tested LLMs exhibit social biases, particularly in less-explored attributes like age, socioeconomic status, and in test case generation tasks.

Key takeaway

For AI Scientists and Machine Learning Engineers developing or deploying code-generating LLMs, you must rigorously evaluate models for social biases beyond common gender and race stereotypes. Focus on less-explored attributes like age, parental income, and educational background, especially in test case generation, where models show increased bias. Implement comprehensive benchmarks like "FairCode" and consider advanced alignment strategies to ensure fairness and inclusivity in your AI systems.

Key insights

LLMs exhibit social biases in code generation, especially in test case generation and less-explored attributes.

Principles

• Bias evaluation requires benchmarks specific to code generation tasks.
• Refusal rate and preference entropy quantify different aspects of bias.
• LLMs show stronger bias in test case generation than function implementation.

Method

FairCode evaluates bias via function implementation (scoring candidates with sensitive attributes) and test case generation (creating biased test cases), quantified by "FairScore" (refusal rate + preference entropy - R*E).

In practice

• Use "FairCode" to benchmark code LLMs for social bias.
• Analyze refusal rates and preference entropy for nuanced bias detection.
• Prioritize bias mitigation for test case generation tasks.

Topics

• LLM Bias
• Code Generation
• FairCode Benchmark
• Social Stereotypes
• Model Evaluation
• AI Ethics

Code references

• YongkDu/FairCode

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Machine Learning Engineer, AI Ethicist

Related on AIssential

• With a Grain of SALT: Are LLMs Fair Across Social Dimensions? — LLM scale does not guarantee fairness; biases against White, Christian, and male-associated outputs are amplified in larger models.
• From If-Statements to ML Pipelines: Revisiting Bias in Code-Generation — Current bias evaluation methods using simple conditionals dramatically underestimate real-world bias in code generation.
• AI is more likely than humans to form biases when hiring — LLMs develop novel biases from experience, stereotyping more than humans due to their generalization optimization.
• A Systematic Analysis of Biases in Large Language Models — LLMs, despite alignment efforts, inherit and perpetuate diverse biases from human-generated training data.
• Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing — Gender bias from LLM writing assistants transfers to human-produced text, particularly suppressing agency in female-targeted writing.
• LLMs are biased — Statistical bias in LLMs is pattern detection, not a flaw; autonomy determines if it scales societal biases.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.