ConfidenceBench: Evaluating Confidence Calibration in Large Language Models

2026-07-24 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

ConfidenceBench is a new calibration benchmark designed to evaluate verbalized confidence estimates in 15 frontier Large Language Models. It uses the Brier score, a proper scoring rule, to incentivize truthful probability reporting without requiring access to model logits, making it applicable to both closed-source and open-source systems. The benchmark consists of 200 private multiple-choice questions across four categories: spatial reasoning, high-precision mathematics, word lookup, and unknowable questions. Across three independent runs, Claude Opus 4.6 and Gemini 3.1 Pro Preview achieved the lowest Brier scores at 0.103, significantly outperforming the calibrated-random baseline of 0.1875. In contrast, Gemini 3.1 Flash-Lite scored 0.367, indicating severe miscalibration. The study found that accuracy and calibration often diverge, highlighting confidence calibration as a distinct and crucial aspect of LLM reliability.

Key takeaway

For MLOps Engineers deploying Large Language Models in critical applications, you should prioritize evaluating verbalized confidence calibration alongside accuracy. Your systems can become more reliable by routing queries with low confidence to human review or stronger models, even if the primary model is highly accurate. This approach mitigates risks from overconfident incorrect answers, ensuring safer and more effective LLM integration. Consider using benchmarks like ConfidenceBench to assess this crucial reliability dimension.

Key insights

LLM confidence calibration is a distinct reliability metric, often diverging from accuracy, crucial for safe deployment.

Principles

• Verbalized confidence is model-agnostic.
• Brier score incentivizes truthful probability.
• Calibration improves with model scale.

Method

ConfidenceBench prompts LLMs for JSON responses (answer A-D, probability 0-100) to 200 private multiple-choice questions across four categories, then evaluates using the Brier score.

In practice

• Use Brier score for LLM uncertainty evaluation.
• Route high-uncertainty queries to stronger models.
• Implement human review for low-confidence answers.

Topics

• LLM Calibration
• Confidence Quantification
• Brier Score
• Verbalized Confidence
• LLM Reliability
• Uncertainty Estimation

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• CALIBER: Calibrating Confidence Before and After Reasoning in Language Models — Confidence in reasoning language models is state-dependent, requiring distinct supervision targets for pre- and post-reasoning estimates.
• A Deep Dive into Calibration of Language Models: Platt Scaling, Isotonic Regression, Temperature Scaling — LLMs are often miscalibrated; post-hoc methods like Temperature Scaling, Platt Scaling, and Isotonic Regression can align confidence with accuracy.
• Bias and Uncertainty in LLM-as-a-Judge Estimation — Bias-corrected LLM-as-a-Judge evaluations are fragile, especially with shared calibration, requiring diagnostics for reliability.
• Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs — LLM uncertainty estimation in multilingual settings is significantly impacted by reasoning language and model scale.
• Probability Calibration: Turning Raw Model Scores Into Confidence You Can Actually Trust — Model accuracy is insufficient; confidence scores must be calibrated to reflect true probabilities for reliable decision-making.
• The Smallest Model Won One of My Tests, and Other Things Benchmarks Won’t Tell You — Real-world AI performance diverges from benchmarks, revealing critical differences in obedience, calibration, and consequence awareness.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.