Learning Credal Ensembles via Distributionally Robust Optimization

2026-06-17 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

CreDRO is a novel approach for quantifying epistemic uncertainty (EU) in deep neural networks, addressing limitations of prior methods that primarily attribute EU to optimization randomness. Developed by Kaizheng Wang at KU Leuven and Nanyang Technological University, CreDRO learns an ensemble of models using distributionally robust optimization (DRO). This method simulates varying degrees of train-test distribution shifts by assigning a range of DRO hyperparameters to individual ensemble members, thereby capturing EU from both training randomness and informative disagreement due to potential distribution shifts. Empirically, CreDRO consistently outperforms state-of-the-art credal classifiers and deep ensemble baselines on out-of-distribution detection benchmarks, including CIFAR10 against SVHN, Places365, CIFAR100, FMNIST, and ImageNet. It also demonstrates improved performance in selective classification tasks within medical settings, such as the Camelyon17 dataset, and shows robustness across different ensemble sizes and hyperparameter choices.

Key takeaway

For machine learning engineers building robust systems, CreDRO offers a superior method for quantifying epistemic uncertainty by explicitly accounting for potential train-test distribution shifts. You should consider integrating CreDRO, particularly in safety-critical applications like medical imaging or autonomous driving, where reliable uncertainty estimates are paramount for tasks like selective classification or out-of-distribution detection. Its consistent empirical outperformance over existing credal and deep ensemble methods suggests it can significantly enhance model trustworthiness and decision-making under real-world data variability.

Key insights

CreDRO quantifies epistemic uncertainty by modeling disagreement from potential train-test distribution shifts via distributionally robust optimization.

Principles

• Epistemic uncertainty should reflect distribution shifts, not just optimization randomness.
• Distributionally robust optimization enhances model robustness to data discrepancies.

Method

CreDRO trains an ensemble using adversarially reweighted learning (ARL-based DRO) with varying hyperparameters (δ_i) to simulate diverse train-test divergences. Inference converts softmax probabilities into box credal sets for EU quantification.

In practice

• Apply CreDRO for improved out-of-distribution detection.
• Utilize box credal sets for efficient EU computation in multi-class problems.
• Consider CreDRO for safety-critical applications requiring reliable uncertainty estimates.

Topics

• Epistemic Uncertainty
• Distributionally Robust Optimization
• Credal Ensembles
• Out-of-Distribution Detection
• Selective Classification
• Deep Learning Robustness

Code references

• timoverse/credal-prediction-relative-likelihood
• kuangliu/pytorch-cifar
• DBO-DKFZ/uncertainty-benchmark

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• CREDO: Epistemic-Aware Conformalized Credal Envelopes for Regression — CREDO combines credal sets and conformal prediction to deliver calibrated, interpretable prediction intervals that explicitly quantify epistemic uncertainty.
• Distributionally Robust Geometric Joint Chance-Constrained Optimization: Neurodynamic Approaches — A neurodynamic duplex neural network solves distributionally robust chance-constrained optimization problems, converging to global optima.
• Bulk-Calibrated Credal Ambiguity Sets: Fast, Tractable Decision Making under Out-of-Sample Contamination — Bulk-calibrated credal ambiguity sets enable tractable, finite worst-case risk in Huber contamination models for robust decision-making.
• Epistemic Uncertainty Is Not the Reducible Kind — The standard definition and measure of epistemic uncertainty are inconsistent, requiring a refined taxonomy.
• Possibilistic Predictive Uncertainty for Deep Learning — DAPPr uses possibility theory for principled, computationally efficient epistemic uncertainty quantification in deep learning.
• Distribution Corrected Offline Data Distillation for Large Language Models — Offline distillation can be improved by correcting teacher-student distribution drift without costly online sampling.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.