Architecture-Adaptive Uncertainty Fusion for Deepfake Detection

2026-06-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, extended

Summary

The Correlation-Optimized Fusion (COF) framework enhances deepfake detection by providing reliable prediction uncertainty, addressing the limitation of existing methods that ignore architecture-specific optimal uncertainty composition. COF fuses five complementary uncertainty sources—epistemic, aleatoric, calibration, conformal, and distributional—by maximizing Pearson correlation between fused scores and prediction errors through constrained optimization on the probability simplex. This architecture-adaptive approach requires no model modifications and optimizes weights in just 42 seconds, significantly faster than a 5-model Deep Ensemble's 20-45 hours. Evaluated across eleven architectures on FaceForensics++, CelebDF, and DFDC, COF achieves a mean in-domain Pearson ρ=0.438. Crucially, under distribution shift, COF outperforms Random Forest in 9/11 architectures on CelebDF with up to 7.3× higher correlation, retaining more signal (74% drop to ρ=0.116) compared to RF's 85% degradation. However, all UQ methods show catastrophic generalization failure cross-dataset, with mean correlation degrading 90.7% and seven architectures exhibiting uncertainty inversion.

Key takeaway

For AI Security Engineers deploying deepfake detectors in forensic contexts, prioritize COF or L1-COF for cross-domain reliability, as they offer significantly higher correlation under distribution shift compared to non-linear methods like Random Forest. You should prefer residual-CNN or larger EfficientNet architectures (e.g., ResNet50, EfficientNet-B4) to mitigate uncertainty inversion risks. Always include conformal prediction in your UQ systems and recognize that feature-space distance is the most stable uncertainty signal across domains. Be aware that all current UQ methods face catastrophic reliability collapse on highly shifted datasets.

Key insights

Correlation-Optimized Fusion (COF) adaptively combines multiple uncertainty sources to predict deepfake detection errors, improving cross-domain reliability.

Principles

• Optimal uncertainty fusion is architecture-dependent.
• Capacity-controlled fusion enhances cross-domain reliability.
• Feature-space distance offers stable cross-domain uncertainty.

Method

COF maximizes Pearson correlation between five normalized uncertainty sources and prediction errors via constrained optimization on the probability simplex using SLSQP, learning architecture-specific weights.

In practice

• Always include conformal prediction in UQ systems.
• Prioritize feature-space distance for cross-domain UQ.
• Use SC-Weight for resource-constrained UQ deployment.

Topics

• Deepfake Detection
• Uncertainty Quantification
• Correlation-Optimized Fusion
• Cross-Domain Generalization
• Conformal Prediction
• Machine Learning Forensics

Code references

• sharmrit/cof-deepfake

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• Architecture-Adaptive Uncertainty Fusion for Deepfake Detection — Correlation-Optimized Fusion adaptively combines five uncertainty sources to improve deepfake detection reliability, especially under distribution shift.
• v266: Proceedings of COPA 2025 — Conformal Prediction provides statistically rigorous uncertainty quantification across diverse machine learning applications and challenges.
• Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates — Restricting uncertainty sampling to the lifting module improves neural operator uncertainty quantification efficiency and accuracy.
• UF-AMA: A unified framework for cross-domain emotion recognition via adaptive multimodal alignment — UF-AMA unifies multimodal physiological signal fusion with adaptive alignment and multi-level domain adaptation for robust cross-domain emotion recognition.
• Breaking the Communication-Accuracy Trade-off: A Sparsified Information Diffusion Framework for Multi-Agent Collaborative Perception — EDC-CIF balances multi-agent tracking accuracy, response speed, and communication efficiency by optimizing local and global filtering.
• UAU-Net: Uncertainty-aware Representation Learning and Evidential Classification for Facial Action Unit Detection — Explicitly modeling uncertainty in both feature representation and classification improves facial Action Unit detection robustness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.