Revisiting the Relation Between Language Model Perplexity and ASR Word Error Rate for Modern End-to-End Speech Recognition

2026-07-06 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

The long-assumed log-log linear relationship between Language Model (LM) perplexity (PPL) and Automatic Speech Recognition (ASR) Word Error Rate (WER) is being re-examined for modern end-to-end ASR systems. These contemporary systems often possess internal language modeling (ILM) capabilities and can integrate neural LMs and Large Language Models (LLMs) through diverse recognition strategies. This research investigates whether external LMs still enhance current end-to-end ASR performance, if the PPL-WER relation remains log-log linear, and how encoder context length influences this dynamic. It also examines how LLM perplexities align with trends observed for standard neural LMs. Crucially, the study demonstrates that ILM subtraction modifies the observed PPL-WER relation, highlighting the necessity of accounting for the decoder's internal LM when assessing external LM quality.

Key takeaway

For ASR developers evaluating system performance or integrating external language models, you must account for the internal language modeling capacity of modern end-to-end systems. Your assessment of external LM quality should involve considering ILM subtraction, as this significantly alters the observed PPL-WER relationship. This ensures a more accurate understanding of how external LMs truly impact your ASR system's Word Error Rate, guiding better model selection.

Key insights

The traditional PPL-WER relationship is complexified by modern ASR's internal LMs and LLM integration, requiring ILM consideration.

Principles

• Modern ASR systems have internal LM capacity.
• External LMs may still improve end-to-end ASR.
• Decoder's internal LM affects external LM quality interpretation.

Method

The paper studies PPL-WER relation by investigating external LM impact, linearity, encoder context length, and LLM perplexities, including ILM subtraction analysis.

In practice

• Consider internal LM capacity in ASR evaluation.
• Evaluate external LMs with ILM subtraction.
• Account for encoder context length effects.

Topics

• Automatic Speech Recognition
• Language Model Perplexity
• Word Error Rate
• End-to-End ASR
• Internal Language Modeling
• Large Language Models

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Analyzing Error Propagation in Korean Spoken QA with ASR-LLM Cascades — ASR-stage information loss, especially single-character errors in Korean, is the primary bottleneck for ASR-LLM cascade performance.
• Evaluation of Automatic Speech Recognition Using Generative Large Language Models — Generative LLMs significantly improve ASR evaluation by aligning better with human perception than traditional WER.
• A Calculus-Based Framework for Determining Vocabulary Size in End-to-End ASR — A calculus-based framework analytically optimizes ASR vocabulary size, outperforming heuristic methods via differentiable cost functions.
• Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6% WER (13.8% cWER) — Benchmark contamination significantly inflates reported ASR performance, especially for dialectal variations.
• Can Voice Agents Handle Bilingual Customers? Benchmarking Frontier ASR on Code-Switched Speech — Frontier ASR models handle code-switched speech with minimal performance degradation, but error patterns vary by language pair and model.
• Contrastive Training with LLM-generated Near-Misses for Robust Code-Switching Speech Recognition — Contrastive training with LLM-generated near-misses improves code-switching ASR by targeting critical regions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.