Low-Latency Turn-Taking via Context-Aware Preface Generation in a Real-World Dialogue Robot

2026-07-25 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

A two-stage incremental framework addresses response delays in large language model (LLM)-based dialogue systems, which typically generate responses only after final speech recognition. This framework decouples prefatory-response preparation from speech onset: an intent readiness detector triggers LLM-based generation of a short preface when user intent is predictable, while a voice activity projection (VAP) model determines its delivery time. A field experiment with a route-guidance robot in a shopping mall evaluated no-filler, fixed-filler, and contextual-preface conditions. Both fixed-filler and contextual-preface significantly reduced initial response latency compared to no-filler. Contextual-preface had longer initial response latency than fixed-filler but a significantly shorter initial-to-main gap, indicating a timing trade-off, with exploratory ratings showing no significant differences.

Key takeaway

For NLP Engineers developing real-time LLM-based dialogue systems, you should consider a two-stage incremental framework to mitigate response delays. By decoupling prefatory response generation from final speech recognition using intent prediction and voice activity projection, you can reduce overall turn-taking latency. Evaluate the trade-off between initial response latency and the gap to the main response in your specific application to optimize user experience.

Key insights

A two-stage framework reduces LLM dialogue latency by decoupling preface generation from final speech recognition.

Principles

• Decouple prefatory response from main generation.
• Predict user intent for early response preparation.
• Balance initial latency with main response gap.

Method

An intent readiness detector triggers LLM-based preface generation, and a voice activity projection (VAP) model determines delivery timing, enabling a two-stage incremental dialogue response.

In practice

• Implement intent readiness detection.
• Integrate VAP for preface delivery.
• Evaluate latency trade-offs in real-world settings.

Topics

• LLM Dialogue Systems
• Low-Latency Dialogue
• Turn-Taking
• Preface Generation
• Voice Activity Projection
• Real-World Robotics

Best for: Research Scientist, AI Scientist, NLP Engineer, Robotics Engineer

Related on AIssential

• TurnNat: Automatic Evaluation of Turn-Taking Naturalness in Dyadic Spoken Dialogue — TurnNat automatically evaluates spoken dialogue turn-taking naturalness using a likelihood-based framework and a causal prediction model.
• Chronological Thinking in Full-Duplex Spoken Dialogue Language Models — Chronological Thinking enables full-duplex SDLMs to reason incrementally during user speech, improving response quality without latency.
• Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging — Decoupled design and trajectory metrics are essential for robust multi-turn LLM evaluation beyond subjective scores.
• Dynamic Commonsense Coordination for Empathetic Response Generation — Dynamic coordination of commonsense across dialogue stages significantly improves empathetic response quality.
• CrossOracle: An Agentic Framework for Real-Time Expert Personas with Parallelized Acoustic Synthesis — CrossOracle optimizes ASR-LLM-TTS pipelines for real-time, low-latency, code-switched voice interaction with expert personas.
• Context-Driven Incremental Compression for Multi-Turn Dialogue Generation — C-DIC uses revisable, per-thread compression states and an incremental update loop to efficiently manage long dialogue histories.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.