Decodable but Not Detectable: A Leakage Fingerprint for Near-OOD Benchmarks

2026-07-23 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

A benchmark leak in near-OOD evaluation, where a trained class is mistakenly designated as "OOD" but remains in the in-distribution (ID) fit set, can lead to severely misleading results, such as unsupervised detectors scoring well below chance (e.g., AUROC 0.326). Correcting this by removing the class from training (leave-class-out protocol) dramatically improves performance to 0.911. The paper introduces a "leak fingerprint" diagnostic, characterized by near-perfect supervised decodability (OracleLDA ≈ 1) coupled with unsupervised detection collapse (best-unsupervised AUROC < 0.65), validated with high sensitivity (18/20) and specificity (31/32) across diverse settings. An audit of 24 standard OOD benchmark pairs confirmed its specificity, firing only on one intrinsically-hard, non-leaked case. Furthermore, the study reveals that perturbation-response OOD detectors, even after protocol correction, offer no significant improvement over plain Mahalanobis-on-embeddings baselines, indicating signals are "decodable but not detectable."

Key takeaway

For AI Scientists and Machine Learning Engineers constructing or evaluating near-OOD benchmarks, you must rigorously verify that designated OOD classes are genuinely absent from the in-distribution fit set. Implement the proposed leak fingerprint, contrasting supervised decodability with unsupervised detection collapse, to cheaply audit splits without retraining. This prevents inverted results and ensures valid OOD detector performance assessments, particularly in embedding space where the diagnostic is most reliable.

Key insights

Benchmark leaks in near-OOD evaluation can invert detector scores, requiring a specific diagnostic to identify contamination.

Principles

• Designate OOD classes genuinely absent from training.
• Supervised decodability contrasts unsupervised collapse.
• Fit-set membership is key to OOD evaluation validity.

Method

Detect benchmark leaks by contrasting near-perfect supervised decodability (OracleLDA ≈ 1) with unsupervised detection collapse (best-unsupervised AUROC < 0.65) in embedding space, without backbone retraining.

In practice

• Audit near-OOD splits for trained classes in fit sets.
• Use the leak fingerprint for cheap, no-retrain diagnostics.
• Prioritize embedding space for leak detection.

Topics

• Out-of-Distribution Detection
• Benchmark Contamination
• Data Leakage
• Near-OOD Evaluation
• Model Auditing
• Leak Fingerprint

Best for: NLP Engineer, Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Out-of-Distribution (OOD) Detectors for Open-Set RF Fingerprinting — OOD detectors for open-set RF fingerprinting can be tuned effectively without auxiliary OOD data, overcoming a key practical barrier.
• Why Your 99% Accurate Model Might Actually Be Useless — Data leakage inflates model accuracy by providing unavailable information, leading to real-world failure.
• Beyond Coordinate Gauge: An Audited Protocol for Detecting Donor-Specific Functional Fingerprints after Neural Collapse — Donor-specific functional fingerprints are detectable in independently trained neural networks after Neural Collapse using affine alignment and baseline correction.
• Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Wins — Overly compact feature clusters, like those from CenterLoss, harm OOD detection by compressing inter-class margins.
• Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring — Reframing test-time canonicalization as OOD detection improves vision model robustness to affine transformations without retraining.
• MM++: Unsupervised Scale-Invariant Multilayer OOD Detection via Top-K Gated Feature Fusion — MM++ enables unsupervised, scale-invariant OOD detection by fusing discriminative intermediate and terminal layer features.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.