Non-Autoregressive Minimum Bayes' Risk Decoding for Fast Speech Recognition

2026-06-16 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A novel Non-Autoregressive Minimum Bayes' Risk (NAR-MBR) decoding framework is proposed to enhance speech recognition speed and accuracy. Non-autoregressive (NAR) decoding generates output tokens in parallel, offering faster processing than sequential autoregressive (AR) methods, but typically suffers from degraded recognition performance due to its inability to condition on previously generated tokens. NAR-MBR addresses this by maximizing expected utility calculated from samples drawn from the NAR model's output probability, rather than just maximizing the probability itself. This approach efficiently obtains multiple samples with a single forward computation. Experiments across LibriSpeech, Switchboard, AMI, and web presentation corpus demonstrated that NAR-MBR decoding outperformed previous NAR decoding methods and ran faster than AR decoding.

Key takeaway

For NLP Engineers and AI Scientists optimizing speech recognition systems, consider NAR-MBR decoding to achieve a critical balance between speed and accuracy. This framework offers faster processing than traditional autoregressive methods while effectively mitigating the performance degradation typically associated with non-autoregressive decoding. You should explore its application for real-time or high-throughput Automatic Speech Recognition (ASR) tasks where both low latency and high fidelity are paramount.

Key insights

NAR-MBR decoding significantly improves non-autoregressive speech recognition accuracy while maintaining high speed.

Principles

• NAR decoding offers speed but lacks context.
• MBR can resolve NAR model uncertainty.
• Parallel sampling boosts efficiency.

Method

NAR-MBR maximizes expected utility from samples drawn from the NAR model's output probability, efficiently obtaining multiple samples with a single forward computation.

In practice

• Implement NAR-MBR for faster ASR.
• Apply MBR to parallel generation tasks.
• Optimize sample generation in NAR models.

Topics

• Speech Recognition
• Non-Autoregressive Decoding
• Minimum Bayes' Risk
• Decoding Algorithms
• ASR Performance
• Parallel Processing

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference — On-device ASR can achieve high accuracy and low latency on CPU through systematic benchmarking and quantization.
• Probing Low Frame Rate Degradation in Neural Audio Codecs — The quality cliff in low frame rate neural audio codecs is a training artifact, not a fundamental limitation.
• Text-Utilization for Encoder-dominated Speech Recognition Models — Efficient text-only data integration improves encoder-dominated speech recognition, favoring simpler models and larger encoders.
• Beyond Waveform Robustness: Robust Feature-Vocoder Adversarial Attacks on Automatic Speech Recognition — ASR adversarial attacks can achieve superior transferability and defense evasion by perturbing self-supervised learning features instead of raw waveforms.
• From A to B to A: Palindromic Zero-Shot Voice Conversion with Non-Parallel Data — A voice conversion framework uses KNN over WavLM to create synthetic training pairs from non-parallel data for zero-shot, multilingual VC.
• Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India — Real-world ASR performance in India requires unscripted data, orthographically-informed metrics, and regional analysis to overcome current benchmark limitations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.