On the QUEST for Uncertainty Quantification via Highest Density Regions

2026-06-17 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new framework called QUEST (Quantifying Uncertainty via highest dEnSiTy regions) addresses limitations in current uncertainty quantification (UQ) for probabilistic machine learning regression problems. Existing scalar UQ methods, often based on proper scoring rules, rely on pointwise predictive risk, which can yield counterintuitive results when the target statistic deviates from the conditional expectation. QUEST proposes characterizing uncertainty by the volume of the most probable subset within a distribution's support. This approach evaluates the concentration of Lebesgue measure at a distribution's peak(s) using a robustness parameter $α$. The authors establish connections between QUEST measures and classical statistics from information theory and economics. Benchmarks on selective prediction confirm QUEST's favorable performance against standard measures like variance and differential entropy, demonstrating its adherence to UQ axioms such as monotonicity under distributional spread and invariance to location shifts.

Key takeaway

For Machine Learning Engineers developing models for safety-critical applications, if you are currently relying on scalar UQ approaches based on pointwise predictive risk, consider evaluating QUEST. This new framework offers a more robust characterization of uncertainty by focusing on highest density regions, satisfying key UQ axioms. Implementing QUEST could lead to more reliable decision-making and improved performance in selective prediction tasks compared to traditional variance or differential entropy measures.

Key insights

QUEST offers a novel UQ framework using highest density regions to overcome pointwise predictive risk limitations.

Principles

• UQ should characterize uncertainty via distribution support volume.
• UQ measures must satisfy axioms like monotonicity and invariance.
• Pointwise predictive risk can be counterintuitive for UQ.

Method

QUEST quantifies uncertainty by evaluating the Lebesgue measure concentration at a distribution's peak(s) within its most probable subset, using a robustness parameter $α$.

In practice

• Apply QUEST in safety-critical ML applications.
• Use highest density regions for robust UQ.
• Benchmark UQ against variance and differential entropy.

Topics

• Uncertainty Quantification
• Probabilistic Machine Learning
• Highest Density Regions
• Regression Models
• Epistemic Uncertainty
• Selective Prediction

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Data Scientist

Related on AIssential

• Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates — Restricting uncertainty sampling to the lifting module improves neural operator uncertainty quantification efficiency and accuracy.
• The Role of Ambiguity in Error Prediction via Uncertainty Quantification — Disentangling input ambiguity from UQ signals significantly improves LLM error prediction, especially for Question Answering tasks.
• Introduction to Deep Evidential Regression for Uncertainty Quantification — Evidential Deep Learning quantifies both epistemic and aleatoric uncertainty in a single inference pass.
• Scalable Uncertainty Quantification for Extreme Weather Forecasting via Empirical Neural Tangent Kernels — NTK-UQ provides scalable, architecture-aware uncertainty quantification for deep learning weather models, outperforming conformal prediction.
• Bulk-Calibrated Credal Ambiguity Sets: Fast, Tractable Decision Making under Out-of-Sample Contamination — Bulk-calibrated credal ambiguity sets enable tractable, finite worst-case risk in Huber contamination models for robust decision-making.
• A Cubing Strategy for Identifying Stable Hyperparameter Regions for Uncertainty Quantification in Spatial Deep Learning — A cubing strategy systematically identifies stable hyperparameter regions for calibrated uncertainty quantification in spatial deep learning with MC dropout.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.