Self-Poisoning in Adaptive Out-of-Distribution Detection: A Sharp-Threshold Theory and Certified Label-Free Calibration

2026-07-27 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Mathematics & Computational Sciences · Depth: Expert, medium

Summary

Test-time adaptive out-of-distribution (OOD) detectors, which update memory banks from unlabelled streams, are prone to "self-poisoning," a phenomenon modeled as a generalized Pólya urn. This theory reveals that when the admission kernel's slope (acting as a reproduction number) is one or greater, the detector's memory bank becomes fully poisoned, leading to collapse and up to a 0.163 AUROC loss. The measured admission kernel is affine (R² ≥ 0.996) with a slope just below one, indicating these detectors are near-critical by design. To counter this, a certified admission gate called WARDEN severs the feedback loop by using conformal e-values against a frozen reserve, preventing poisoning and maintaining a mean realized FPR of 0.056 with dictionary impurity never exceeding 0.111. Additionally, the paper introduces CDC, a label-free method that restores nominal FPR under ID drift, addressing static-calibration failures. An impossibility theorem demonstrates that label-free procedures face a power ceiling due to the indistinguishability of drift and contamination, a limit CDC approaches by retaining 63-67% of oracle power.

Key takeaway

For Machine Learning Engineers deploying adaptive OOD detectors, recognize that these systems are inherently near-critical and prone to self-poisoning, leading to performance collapse. You should implement certified admission mechanisms like WARDEN to sever feedback loops, ensuring dictionary impurity remains low and FPR is controlled. Additionally, consider CDC for label-free calibration to maintain nominal FPR under ID drift, accepting the inherent power ceiling when distinguishing drift from contamination without labels.

Key insights

Adaptive OOD detectors are inherently unstable due to self-poisoning, but certified admission can prevent collapse and maintain performance.

Principles

• Self-poisoning in OOD detection is a predictable phase transition.
• Feedback loops in admission evidence cause detector collapse.
• Label-free methods face power limits distinguishing drift/contamination.

Method

WARDEN severs the feedback loop by using conformal e-values against a frozen reserve, gated by e-BH (e ≥ 1/δ). CDC calibrates thresholds on contaminated streams using a contamination-corrected quantile and DKW slack.

In practice

• Implement WARDEN to cap OOD dictionary impurity below 0.111.
• Apply CDC to restore nominal FPR under ID drift.

Topics

• Out-of-Distribution Detection
• Test-Time Adaptation
• Data Poisoning
• Conformal Prediction
• WARDEN
• ID Drift

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents — Out-of-band defenses for LLM agents show promise against adaptive prompt injection, but require robust, dynamic evaluation.
• Certified Robustness to Data Poisoning in Gradient-Based Training — A framework provides provable guarantees against data poisoning and backdoor attacks in gradient-based machine learning models.
• Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software — LLMs fine-tuned for vulnerability detection achieve "calibration without comprehension," adapting outputs without true security reasoning.
• Widening the Gap: Exploiting LLM Quantization via Outlier Injection — Outlier injection during quantization can predictably induce malicious behavior in LLMs, even with advanced schemes.
• Sequential Auditing for f-Differential Privacy — Sequential f-DP auditors adaptively determine sample sizes, significantly reducing auditing costs and improving privacy violation detection.
• Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs — Combining OOD detectors with guard models significantly improves LLM safety monitoring for unforeseen alignment failures.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.