Suppressing Unnecessary Clarification Requests for Unknown Word Acquisition in Spoken Dialogue Using Syllable-Based ASR Confidence

2026-07-20 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new approach addresses unnecessary clarification requests in spoken dialogue systems when acquiring unknown words. The problem arises because syllable-based automatic speech recognition (S-ASR) errors can misidentify known words as unknown, leading to excessive user burden. This method extends a reinforcement learning policy state with recognition reliability features, specifically two confidence measures derived from S-ASR. It also incorporates segmentation confidence over N-best hypotheses to mitigate minor S-ASR errors. Experiments using pre-recorded speech data demonstrated a reduction of 1.34 in clarification requests for utterances affected by S-ASR errors. Furthermore, the area under the learning curve for word segmentation numerically increased by 0.07.

Key takeaway

For NLP Engineers developing spoken dialogue systems focused on unknown word acquisition, you should integrate syllable-based ASR confidence and N-best segmentation confidence into your reinforcement learning policy. This approach significantly reduces unnecessary clarification requests caused by ASR errors, improving user experience and system efficiency. By making your system's clarification decisions sensitive to recognition reliability, you can streamline the word acquisition process.

Key insights

Integrating ASR and segmentation confidence into dialogue policy effectively reduces unnecessary clarification requests for unknown words.

Principles

• Syllable-based ASR errors can mimic unknown words, causing false clarification requests.
• Reinforcement learning policy states can be enhanced with recognition reliability features.
• N-best hypotheses improve segmentation confidence, reducing minor ASR error impact.

Method

Extend a reinforcement learning policy state with two syllable-based ASR confidence measures and segmentation confidence over N-best hypotheses to filter clarification requests.

In practice

• Incorporate syllable-based ASR confidence scores into your dialogue system's policy.
• Utilize N-best ASR hypotheses to derive more robust word segmentation confidence.

Topics

• Spoken Dialogue Systems
• Automatic Speech Recognition
• Reinforcement Learning
• Unknown Word Acquisition
• Syllable-Based ASR
• Confidence Measures
• Active Learning

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Analyzing Error Propagation in Korean Spoken QA with ASR-LLM Cascades — ASR-stage information loss, especially single-character errors in Korean, is the primary bottleneck for ASR-LLM cascade performance.
• AST: Adaptive, Seamless, and Training-Free Precise Speech Editing — AST enables training-free, precise speech editing by manipulating latent spaces of pre-trained autoregressive TTS models.
• Clicky Meets Universal-3.5: Voice-Navigate Your Code — Dynamic key term updating from screen content significantly enhances speech-to-text accuracy for domain-specific vocabulary, enabling precise voice navigation.
• Intent vs. Surface: Recovering Acoustic Realization from Modern ASR for Pronunciation Training — Modern ASR's intent bias masks mispronunciations, but surface-faithful reranking improves error detection for language learners.
• Automatic Speech Recognition for Child Reading: A Phonemic Approach using Isolated Words in Brazilian Portuguese — A phonemic approach with fine-tuned Wav2Vec2.0 significantly improves ASR for child reading assessment in Brazilian Portuguese.
• Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech — A diarization-guided, multi-stage ASR adaptation system significantly improves multilingual two-speaker conversational speech recognition.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.