Concerning Uncertainty -- A Systematic Survey of Uncertainty-Aware XAI

2026-03-31 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

A systematic survey of uncertainty-aware explainable artificial intelligence (UAXAI) analyzes how uncertainty is integrated into AI explanation pipelines and evaluated. The survey identifies three primary approaches to uncertainty quantification: Bayesian, Monte Carlo, and Conformal methods. It also categorizes strategies for incorporating uncertainty into explanations, including assessing trustworthiness, constraining models or explanations, and explicitly communicating uncertainty. Evaluation practices for UAXAI remain fragmented, largely focusing on model-centric metrics rather than user-centered assessments, with inconsistent reporting of reliability properties like calibration and explanation stability. Recent trends, particularly from 2023-2025, show a shift towards calibration, distribution-free techniques, and a growing recognition of explainer variability. The study emphasizes the need for unified evaluation principles that connect uncertainty propagation, robustness, and human decision-making, highlighting counterfactual and calibration approaches as promising for aligning interpretability with reliability.

Key takeaway

For research scientists developing or deploying AI systems in high-stakes environments, understanding UAXAI is crucial for building trustworthy applications. You should focus on integrating formal uncertainty quantification methods like conformal prediction and Bayesian approaches into your explanation pipelines. Prioritize evaluating not just model performance, but also explanation stability, calibration, and how different uncertainty representations influence user trust and decision quality, moving beyond purely model-centric metrics.

Key insights

UAXAI integrates uncertainty into AI explanations to improve transparency, reliability, and human trust in high-stakes decision-making.

Principles

• Uncertainty is a multifaceted concept that can be classified, measured, and communicated.
• Uncertainty can accumulate across the entire socio-technical AI pipeline.
• UAXAI aims to clarify when and why explanations can be trusted.

Method

UAXAI methods either treat uncertainty as an explanatory signal, augment XAI techniques with uncertainty estimates, or use uncertainty to evaluate explanation robustness and stability.

In practice

• Use conformal prediction for formal uncertainty guarantees.
• Decompose uncertainty into aleatoric and epistemic components.
• Prioritize user studies for evaluating uncertainty representations.

Topics

• Uncertainty-Aware XAI
• Uncertainty Quantification
• Aleatoric Uncertainty
• Epistemic Uncertainty
• Bayesian Methods

Code references

• Moffran/UAXAI_lsurvey

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Distribution-Free Uncertainty Quantification for Continuous AI Agent Evaluation — Conformal prediction offers robust, distribution-free uncertainty quantification for continuous AI agent evaluation.
• Uncertainty, Vagueness, and Ambiguity in Human-Robot Interaction: Why Conceptualization Matters — Consistent conceptualization of uncertainty, vagueness, and ambiguity is crucial for advancing human-robot interaction.
• Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates — Restricting uncertainty sampling to the lifting module improves neural operator uncertainty quantification efficiency and accuracy.
• Quantifying and Understanding Uncertainty in Large Reasoning Models — A new method quantifies LRM uncertainty with statistical guarantees, linking reasoning traces to final answers.
• CREDO: Epistemic-Aware Conformalized Credal Envelopes for Regression — CREDO combines credal sets and conformal prediction to deliver calibrated, interpretable prediction intervals that explicitly quantify epistemic uncertainty.
• UAU-Net: Uncertainty-aware Representation Learning and Evidential Classification for Facial Action Unit Detection — Explicitly modeling uncertainty in both feature representation and classification improves facial Action Unit detection robustness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.