Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Wins

2026-05-23 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

The Geometry-Optimised Epistemic Network (GOEN) is a novel pipeline designed to enhance out-of-distribution (OOD) detection for safe machine learning deployment. GOEN integrates multi-scale features from a ResNet-18 backbone, L2 normalisation, Mahalanobis distance, and a calibration head trained with real hard OOD examples. A critical finding is that CenterLoss, a common regulariser for feature compactness, significantly degrades OOD detection performance, reducing average OOD AUROC from 0.9483 to 0.9366, despite improving classification accuracy. The superior variant, GOEN-NoCenterLoss, achieves an average OOD AUROC of 0.9483 on CIFAR-10 benchmarks, surpassing baselines like deep ensembles (0.8827), KNN (0.8967), and ODIN (0.8870). GOEN is efficient, training in under 20 minutes on a single GPU, providing a practical solution for reliable OOD recognition.

Key takeaway

For Machine Learning Engineers building safety-critical AI systems, this research indicates that optimizing solely for classification accuracy can compromise out-of-distribution detection. You should re-evaluate the use of feature compactness regularizers like CenterLoss, as they can degrade epistemic uncertainty. Instead, consider adopting multi-scale feature extraction, L2 normalization, and Mahalanobis distance, calibrated with diverse OOD examples, to build models that reliably recognize their own limitations and improve system trustworthiness.

Key insights

Overly compact feature clusters, like those from CenterLoss, harm OOD detection by compressing inter-class margins.

Principles

• Classification and epistemic uncertainty objectives are not inherently aligned.
• Multi-scale features provide complementary OOD signals for diverse distribution shifts.
• L2 normalization stabilizes covariance estimation for effective distance-based OOD detection.

Method

GOEN trains a multi-scale ResNet-18 backbone (without CenterLoss), fits class-conditional Mahalanobis densities on L2-normalized features, then trains a calibration head using in-distribution and hard OOD examples.

In practice

• Avoid CenterLoss when OOD detection is a primary safety concern.
• Incorporate multi-scale features to detect both texture-level and semantic-level shifts.
• Calibrate OOD detectors with real-world hard OOD examples for better generalization.

Topics

• Out-of-Distribution Detection
• Epistemic Uncertainty
• Feature Geometry
• Mahalanobis Distance
• CenterLoss
• Multi-Scale Features
• Model Calibration

Best for: Research Scientist, Computer Vision Engineer, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• MM++: Unsupervised Scale-Invariant Multilayer OOD Detection via Top-K Gated Feature Fusion — MM++ enables unsupervised, scale-invariant OOD detection by fusing discriminative intermediate and terminal layer features.
• Learning Credal Ensembles via Distributionally Robust Optimization — CreDRO quantifies epistemic uncertainty by modeling disagreement from potential train-test distribution shifts via distributionally robust optimization.
• Normalization vs Standardization — Normalization and standardization differ significantly in outlier sensitivity, guiding their appropriate use.
• Infinite-dimensional Mahalanobis Distance with Applications to Kernelized Novelty Detection — The Mahalanobis distance is extended to infinite-dimensional spaces via a variance norm for enhanced novelty detection.
• Entropy Centroids as Intrinsic Rewards for Test-Time Scaling — Temporal patterns of LLM uncertainty, captured by Entropy Centroids, predict response quality without external rewards.
• Component-Based Out-of-Distribution Detection — CoOD detects OOD inputs by analyzing local component shifts and compositional inconsistencies, improving detection granularity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.