Learning to Say “I Don’t Know”: The First Floor of Awareness

2026-07-14 · Source: Towards AI - Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Advanced, extended

Summary

This essay, part of the "Humble Model Series," introduces the "first floor" of an AI awareness architecture, enabling models to express uncertainty and "say 'I don't know'." It implements three mechanisms: Monte Carlo Dropout, Deep Ensembles, and Out-of-Distribution (OOD) Detection, integrating them into a decision gate. Experiments on MNIST and Fashion-MNIST showed MC Dropout variance rising 31x and Deep Ensemble variance 28x from clean to OOD data, compared to 6x and 4x for adversarial inputs. A class-conditional Mahalanobis distance achieved a Cohen's d of 0.99 for OOD detection. The fused decision gate, evaluated on 10,000 images per set, deferred 2.52% of clean MNIST inputs (achieving 99.85% accuracy on predictions) and 80.09% of Fashion-MNIST inputs. While fusion reduced silent OOD failures by 61% (from ~1,878 to ~724), it decreased clean data coverage from 97.48% to 87.04%. Critically, the gate deferred only 8.80% of FGSM-perturbed adversarial inputs (ε=0.25), revealing a significant gap in adversarial awareness compared to distributional unfamiliarity.

Key takeaway

For Machine Learning Engineers deploying models in environments sensitive to mispredictions, you should recognize that current uncertainty quantification and OOD detection methods, while effective for natural distribution shifts, offer limited protection against adversarial attacks. Your awareness architecture must explicitly address adversarial robustness, as ensemble disagreement and feature-space distances do not reliably detect such perturbations. Prioritize research and implementation of techniques specifically designed for adversarial awareness to prevent silent, confident failures.

Key insights

AI models can be engineered to express uncertainty and defer, but current methods struggle with adversarial inputs.

Principles

• Softmax outputs are not calibrated uncertainty.
• Model awareness needs calibration and a gate.
• Adversarial and OOD awareness are distinct.

Method

Combine Monte Carlo Dropout or Deep Ensembles with class-conditional feature-space OOD detection. Integrate these signals into a threshold-based decision gate to defer uncertain predictions.

In practice

• Implement MC Dropout for epistemic uncertainty.
• Use Deep Ensembles for prediction variance.
• Apply class-conditional Mahalanobis distance.

Topics

• Uncertainty Quantification
• Out-of-Distribution Detection
• Adversarial Robustness
• Deep Ensembles
• Monte Carlo Dropout
• Decision Gates
• Model Awareness

Code references

• Maee127/Adversarial-Notebooks

Best for: AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning — Evaluating epistemic uncertainty by its ability to identify regret offers a more robust decision-theoretic framework than proxy tasks.
• Frontier Models Think I'm Eight Different People — AI model "recognition" scores often reflect fluent guessing, not true knowledge, especially when models cannot admit ignorance.
• Calibrating Uncertainty for Zero-Shot Adversarial CLIP — Adversarial attacks on CLIP suppress uncertainty, leading to over-confident misclassifications; calibrating this is crucial for reliability.
• Plug-in Losses for Evidential Deep Learning: A Simplified Framework for Uncertainty Estimation that Includes the Softmax Classifier — A simplified Evidential Deep Learning framework uses plug-in losses to achieve reliable, efficient uncertainty estimation, integrating softmax for broader applicability.
• Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks? — Deep ensembles for GNNs suffer "epistemic collapse," limiting their uncertainty quantification benefits due to lack of diversity.
• Learning to be uncertain before learning from data — Training neural networks on random noise before data exposure improves calibration and reliability.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards AI - Medium.