NeMo@IWSLT 2026: Cascaded System for Simultaneous Speech Translation

2026-06-29 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

The NVIDIA NeMo team submitted a cascaded system to the IWSLT 2026 Simultaneous Speech Translation (SimulST) tracks. This architecture integrates a dual-mode Unified ASR Transducer model with a multilingual Large Language Model (LLM). The ASR component is specifically trained to produce stable transcriptions across a broad range of latencies, establishing a reliable input for high-quality LLM translation. The submission covers English–German, English–Italian, and English–Chinese tasks in both standard and contextualized settings, alongside the Czech–English standard track, addressing both low- and high-latency scenarios. The team also analyzed how ASR and LLM design choices influence the system's overall latency and translation quality.

Key takeaway

For Machine Learning Engineers developing real-time translation systems, this cascaded ASR-LLM approach offers a robust blueprint. You should prioritize ASR model training for transcription stability across varying latencies, as this directly underpins the LLM's translation quality. Consider implementing a dual-mode ASR to effectively manage both low- and high-latency simultaneous speech translation requirements for your target language pairs.

Key insights

A cascaded ASR-LLM system achieves stable, high-quality simultaneous speech translation across diverse languages and latencies.

Principles

• Stable ASR is foundational for LLM translation quality.
• Dual-mode ASR supports varied latency requirements.
• Cascaded systems can combine specialized models.

Method

A cascaded architecture combines a dual-mode Unified ASR Transducer, trained for stable transcriptions across latencies, with a multilingual Large Language Model for translation.

In practice

• Simultaneous English-German translation.
• Simultaneous English-Italian translation.
• Simultaneous Czech-English translation.

Topics

• Simultaneous Speech Translation
• Cascaded Systems
• ASR Transducer
• Large Language Models
• IWSLT 2026
• Low-Latency Translation
• Multilingual Models

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Analyzing Error Propagation in Korean Spoken QA with ASR-LLM Cascades — ASR-stage information loss, especially single-character errors in Korean, is the primary bottleneck for ASR-LLM cascade performance.
• AlignAtt4LLM: Fast AlignAtt for Decoder-Only LLMs at IWSLT 2026 Simultaneous Speech Translation Task — Adapting AlignAtt for decoder-only LLMs enables simultaneous speech translation by re-engineering attention mechanisms.
• NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR — NIM4-ASR optimizes LLM-based ASR for efficiency and robustness by redesigning multi-stage training and integrating production-ready features.
• Beyond Cascaded Pipelines: Building a Native Spoken Language Model Prototype — Native Spoken Language Models (SLMs) overcome cascaded pipeline limitations by directly processing audio, preserving nuance, reducing errors, and lowering latency.
• MLLP-VRAIN UPV system for the IWSLT 2026 Simultaneous Speech Translation task — The system combines Parakeet and Qwen 3.5 with adaptive policies and RAG for improved SimulST quality and latency.
• Leveraging Audio-LLMs to Filter Speech-to-Speech Training Data — An audio-LLM can effectively filter noisy speech-to-speech translation data using a Rank-to-Distill strategy, improving S2ST performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.