What You Train Is What You Get: Gender Bias, Training Composition, and Post-Hoc Mitigation in Audio Deepfake Detection

2026-07-10 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Audio deepfake detection models often exhibit significant performance disparities across demographic groups despite high overall accuracy. This work investigates gender bias in these models using the ASVspoof5 dataset, employing a controlled custom split to isolate gender-composition effects. Researchers trained attack-specific ResNet18 classifiers, utilizing both LogSpectrogram and WavLM-Base+ features, on nine training sets with diverse gender compositions. Experimental results reveal that training data composition strongly predicts bias direction, with the underrepresented gender consistently performing worse at test time. Notably, WavLM-Base+ features generated gender performance gaps 3.0 to 4.3 times larger than LogSpectrogram under identical conditions. While balanced training reduced LogSpectrogram bias, it left WavLM bias largely intact. Furthermore, all six evaluated post-hoc threshold calibration methods, including Oracle calibration, failed to alter the Equal Error Rate gap, which remained at 1.317 pp, indicating their inability to correct underlying score distribution disparities. These findings emphasize that gender fairness in audio deepfake detection requires intervention during the training phase, as post-hoc solutions offer only partial mitigation.

Key takeaway

For Machine Learning Engineers developing audio deepfake detection systems, you must prioritize addressing gender fairness during the training phase. Your choice of training data composition directly dictates bias direction, with underrepresented genders performing worse. Furthermore, feature selection significantly impacts bias magnitude; WavLM-Base+ features, for instance, produced 3.0 to 4.3 times larger gaps than LogSpectrogram. Relying on post-hoc threshold calibration is insufficient, as these methods cannot correct fundamental score distribution disparities, leaving Equal Error Rate gaps unchanged. Therefore, ensure your training data is balanced and carefully select features to build inherently fairer models.

Key insights

Training data composition dictates gender bias in audio deepfake detection, rendering post-hoc mitigation ineffective.

Principles

• Underrepresented genders perform worse in biased training sets.
• Feature choice significantly impacts bias magnitude.
• Post-hoc calibration cannot fix fundamental score distribution disparities.

Method

Train attack-specific ResNet18 models on ASVspoof5 with custom gender-composition splits, using LogSpectrogram and WavLM-Base+ features, then evaluate post-hoc threshold calibration.

In practice

• Prioritize balanced training data for gender fairness.
• Carefully select features to minimize inherent bias.

Topics

• Audio Deepfake Detection
• Gender Bias
• Training Data Composition
• Post-Hoc Mitigation
• ResNet18
• WavLM-Base+ Features

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, AI Ethicist

Related on AIssential

• Probing Speaker Identity Sensitivity in Audio Deepfake Detectors — Audio deepfake detector failures often stem from speaker identity reliance, quantifiable by the Identity Sensitivity Score (ISS).
• Echoes: A semantically-aligned music deepfake detection dataset — Semantic alignment and provider diversity are crucial for robust AI music deepfake detection.
• Explainable-by-Design Audio Deepfake Detection via Wiener-Hopf Linear Prediction — Explainable audio deepfake detection is achievable using Wiener-Hopf linear prediction and a lightweight CNN, revealing acoustic inconsistencies in synthetic speech.
• Evaluating and Mitigating Gender Bias in Pre-trained Embeddings for ML-based Recruitment — Gender bias persists in ML recruitment embeddings even after scrubbing, requiring multi-faceted mitigation.
• VendorBench-100: A Unified Cross-Paradigm Benchmark for Deepfake Image Detection — Deepfake detection performance varies significantly across paradigms, with metric divergence highlighting unreliable default decision thresholds.
• Behind the Laughter: Uncovering Gender Bias in Code-Mixed Bangla Memes — A multimodal framework effectively detects gender bias in code-mixed Bangla memes using textual and visual analysis.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.