QC-GAN: A Parameter-Efficient Quaternion Conformer GAN for High-Fidelity Speech Enhancement

2026-06-18 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Speech Processing · Depth: Expert, extended

Summary

The Quaternion Conformer GAN (QC-GAN) is a new parameter-efficient speech enhancement framework that integrates a Quaternion Conformer generator with MetricGAN-based training. It leverages the Hamilton product for structured weight sharing, which encodes magnitude and phase information jointly, significantly reducing layer parameters while maintaining their interdependencies. A metric-learning discriminator is employed to maximize perceptual quality by optimizing approximate perceptual evaluation scores. On the VoiceBank+DEMAND dataset, QC-GAN (Base) achieved a Perceptual Evaluation of Speech Quality (PESQ) score of 3.48 with only 0.89M parameters, delivering performance comparable to leading models at less than half their size. An ultra-compact 35K-parameter variant, QC-GAN (Tiny), achieved a PESQ score of 3.23, surpassing conventional lightweight methods. Evaluation on the DNS-Challenge 3 dataset further confirmed its generalization to real-world conditions, with QC-GAN (Base) achieving the highest DNSMOS OVRL (2.73), BAK (3.79), and P.808 MOS (3.37) scores. Ablation studies demonstrated that quaternion representations improved phase reconstruction, reducing group delay error by approximately 9%.

Key takeaway

For Machine Learning Engineers developing lightweight speech enhancement solutions, QC-GAN presents a compelling architecture. You should consider integrating Quaternion Neural Networks and MetricGAN-based training to achieve high perceptual quality with significantly fewer parameters. This approach can reduce your model size by over 50% while maintaining or improving PESQ scores. Explore fused quaternion kernels or linear attention to further optimize inference latency for real-time CPU deployment.

Key insights

Quaternion Conformer GANs achieve high-fidelity speech enhancement with significantly fewer parameters by jointly modeling magnitude and phase.

Principles

• Hamilton product reduces parameters by 75% via structured weight sharing.
• Quaternion algebra provides inductive bias for coupled magnitude-phase encoding.
• MetricGAN training optimizes for human perceptual quality metrics.

Method

QC-GAN employs a Quaternion Conformer generator and a metric discriminator, optimizing a multi-task loss including differentiable PESQ.

In practice

• Apply quaternion layers for parameter-efficient audio processing.
• Integrate MetricGAN for perceptually-driven model training.
• Map STFT magnitude and phase directly to quaternion components.

Topics

• Speech Enhancement
• Quaternion Neural Networks
• Conformer
• MetricGAN
• Parameter Efficiency
• Phase Reconstruction

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• QC-GAN: A Parameter-Efficient Quaternion Conformer GAN for High-Fidelity Speech Enhancement — QC-GAN offers high-fidelity speech enhancement with significantly fewer parameters by combining Quaternion Conformers and MetricGAN training.
• ART: A Faster Way to Train Speech Recognition Models — ART accelerates RNN-T training by combining CTC for baseline alignment with residual learning for refinement.
• Self-Guidance: Enhancing Neural Codecs via Decoder Manifold Alignment — Self-Guidance enhances neural codecs by aligning decoder feature manifolds, reducing quantization error and simplifying token spaces for speech LLMs.
• BASENet: Band-Adapted Speech Enhancement Network with Cross-Band Attention — BASENet enhances speech by adapting network capacity to Bark-scale frequency bands and using cross-band attention for efficient, perceptually-aware processing.
• In Silico Modeling of the RAMPHO Buffer: Dissociating Informational and Energetic Masking via Phonetic Entropy in Deep Neural Networks — Modeling phonetic entropy in DNNs can dissociate informational from energetic masking in multi-talker environments.
• Fourier Multi-Component and Multi-Layer Neural Networks: Unlocking High-Frequency Potential — FMMNNs combine MMNN structure with sine/SinTU activations for superior high-frequency approximation and efficient training.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.