Diffusion Language Models for Speech Recognition

2026-04-16 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

A new paper, "Diffusion Language Models for Speech Recognition," explores the application of diffusion language models (DLMs) to enhance speech recognition accuracy. Authors Davyd Naveriani, Albert Zeyer, Ralf Schlüter, and Hermann Ney introduce a guide for integrating masked diffusion language models (MDLM) and uniform-state diffusion models (USDMs) for rescoring Automatic Speech Recognition (ASR) hypotheses. The work also details a novel joint-decoding method that combines Connectionist Temporal Classification (CTC) with USDM. This method integrates framewise probability distributions from CTC with labelwise probability distributions from USDM at each decoding step, generating new candidates that leverage both strong language knowledge and acoustic information. The findings indicate that both USDM and MDLM significantly improve the accuracy of recognized text, with all code and recipes publicly available.

Key takeaway

For AI Engineers and Research Scientists working on speech recognition systems, integrating diffusion language models like USDM or MDLM can substantially boost text accuracy. You should consider implementing the proposed joint-decoding method, which combines CTC's acoustic information with USDM's language knowledge, to generate more robust ASR candidates. The availability of open-source code and recipes facilitates immediate experimentation and deployment in your projects.

Key insights

Diffusion language models significantly improve speech recognition accuracy through novel rescoring and joint-decoding methods.

Principles

• Bidirectional attention enhances text generation.
• Combining acoustic and language models improves ASR.

Method

The proposed method integrates CTC's framewise probabilities with USDM's labelwise probabilities during decoding to generate new ASR candidates, combining acoustic and language information for improved accuracy.

In practice

• Use MDLM or USDM for ASR hypothesis rescoring.
• Implement joint-decoding with CTC and USDM.
• Access published code and recipes for implementation.

Topics

• Diffusion Language Models
• Speech Recognition
• ASR Hypothesis Rescoring
• Joint Decoding
• Uniform-State Diffusion Models

Best for: AI Engineer, Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Diffusion Language Models for Speech Recognition — Diffusion language models significantly enhance speech recognition accuracy through rescoring and joint-decoding methods.
• A Survey on Diffusion Language Models — Diffusion Language Models (DLMs) achieve parallel text generation and bidirectional context through iterative denoising, rivaling autoregressive models in performance.
• Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL [R] — MDLMs' any-order denoising objective enables globally coherent text generation, outperforming autoregressive models in agentic RL world modeling.
• Continuous Language Diffusion as a Decoder-Interface Problem — Continuous language diffusion models succeed by guiding latent states into "decoder-basin" regions where decoders can reliably interpret tokens.
• Accelerating Portuguese Masked Diffusion Models through Representation Alignment — REPA accelerates Portuguese MDLM training and improves perplexity by aligning internal representations with teacher encoders.
• Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models — Attention-guided denoising and optimization (AGDO) significantly enhances dLLM reasoning by leveraging intrinsic token dependencies.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.