ADAPT–MTU HAI at IWSLT2026: Robust Cascaded Speech Translation for Bhojpuri–Hindi and Irish–English

2026-06-29 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

The ADAPT–MTU HAI team submitted a robust cascaded speech translation framework to the IWSLT 2026 Low-Resource Speech Translation task, addressing challenges like limited data and error propagation. This framework integrates Whisper-based ASR with NLLB-200 multilingual translation for Bhojpuri→Hindi and Irish→English language pairs. For Bhojpuri→Hindi, the optimal configuration, utilizing Whisper-large-v3 and direct NLLB, achieved a BLEU score of 25.59, chrF++ of 42.48, and TER of 63.83 on the full development set, surpassing pivot and copy baselines. In the Irish→English task, replacing Whisper with a language-specific Wav2Vec2 ASR model significantly improved ASR coverage from 94.8% to 100% on the test set, while maintaining low repetition rates. The findings underscore the critical impact of ASR quality on subsequent translation performance, the situational advantages of pivot translation, and the overall efficacy of modular cascaded architectures for low-resource speech translation.

Key takeaway

For NLP Engineers developing low-resource speech translation systems, you should prioritize robust ASR components, as their quality directly dictates downstream machine translation performance. Consider starting with Whisper-large-v3 for ASR, but be prepared to integrate language-specific models like Wav2Vec2 if coverage issues arise. When designing your translation routing, evaluate both direct NLLB-200 and pivot-based strategies, as pivot benefits are conditional. This modular approach can significantly improve accuracy for challenging language pairs.

Key insights

Cascaded ASR-MT systems effectively address low-resource speech translation, with ASR quality being paramount.

Principles

• ASR quality critically impacts downstream MT.
• Pivot translation offers conditional benefits.
• Modular cascaded architectures are effective.

Method

A cascaded framework combines Whisper-based ASR and NLLB-200 multilingual translation, evaluating direct and pivot-based routing strategies for low-resource language pairs.

In practice

• Use Whisper-large-v3 for ASR in similar setups.
• Consider language-specific Wav2Vec2 for ASR.
• Evaluate direct vs. pivot translation routing.

Topics

• Speech Translation
• Low-Resource Languages
• ASR
• Machine Translation
• Cascaded Architectures
• Whisper Model
• NLLB-200

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• BaltiVoice: A Speech Corpus and Fine-tuned Whisper ASR System for the Balti Language — Fine-tuning pre-trained ASR models with modest, newly created corpora can establish functional speech recognition for low-resource languages.
• Hy-MT2: A Family of Fast, Efficient and Powerful Multilingual Translation Models in the Wild — Hy-MT2 delivers high-quality, efficient multilingual translation with strong instruction-following across diverse real-world and resource-constrained scenarios.
• Streaming Speech-to-Text Translation with a SpeechLLM — A novel SpeechLLM adaptively decides when to emit translation tokens, achieving low latency and high quality.
• PiDA: Phonetically-Informed Data Augmentation for Robust Vietnamese Speech Translation — Phonetically-informed data augmentation significantly improves Vietnamese speech translation robustness against ASR errors.
• Can Voice Agents Handle Bilingual Customers? Benchmarking Frontier ASR on Code-Switched Speech — Frontier ASR models handle code-switched speech with minimal performance degradation, but error patterns vary by language pair and model.
• Multilingual Reasoning Cascades Need More Context — Translation cascades benefit significantly from preserving original language context throughout the pipeline.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.