IIIT-BGP IWSLT 2026 Systems for Low-resource ST

2026-06-29 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

The IIIT-BGP team developed low-resource Bhojpuri-Hindi speech translation systems for the IWSLT 2026 shared task, exploring both end-to-end and cascaded architectures. Their end-to-end model integrates a Bhojpuri-finetuned Wav2Vec2 encoder with a pretrained NLLB-200 decoder. This connection is facilitated by a lightweight interconnection adapter, which incorporates learnable layer aggregation, CNN-based temporal compression, and Transformer refinement, with an option for LoRA-based decoder adaptation. For the cascaded approach, the team finetuned Whisper for Bhojpuri Automatic Speech Recognition (ASR) and NLLB-200 for Hindi Machine Translation (MT). This cascaded system further employs QE Fusion with COMET-Kiwi to enhance translation selection from beam candidates, aiming to improve overall translation quality in low-resource settings.

Key takeaway

For NLP Engineers developing low-resource speech translation systems, consider integrating both end-to-end and cascaded architectures. You should explore lightweight interconnection adapters for combining pretrained models like Wav2Vec2 and NLLB-200, potentially with LoRA. Additionally, for cascaded setups, finetuning Whisper for ASR and NLLB-200 for MT, then applying QE Fusion with COMET-Kiwi, can significantly improve translation quality by refining beam search outputs. This approach offers robust strategies for challenging language pairs.

Key insights

Hybrid end-to-end and cascaded systems, leveraging adapter-based integration and quality estimation, enhance low-resource speech translation.

Principles

• Combining pre-trained models with lightweight adapters is effective.
• Cascaded systems benefit from finetuning ASR and MT components.
• QE Fusion can refine beam search outputs in cascaded ST.

Method

The end-to-end method connects a finetuned Wav2Vec2 encoder to an NLLB-200 decoder via an adapter. The cascaded method finetunes Whisper for ASR and NLLB-200 for MT, applying QE Fusion with COMET-Kiwi for translation selection.

In practice

• Use Wav2Vec2 and NLLB-200 for low-resource ST.
• Implement lightweight adapters for model interconnection.
• Apply QE Fusion with COMET-Kiwi for beam candidate selection.

Topics

• Low-resource Speech Translation
• End-to-end Speech Translation
• Cascaded Speech Translation
• Wav2Vec2
• NLLB-200
• COMET-Kiwi
• Adapter Layers

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Giving Voice to the Constitution: Low-Resource Text-to-Speech for Quechua and Spanish Using a Bilingual Legal Corpus — Cross-lingual transfer mitigates data scarcity in low-resource languages for high-quality TTS.
• MLLP-VRAIN UPV system for the IWSLT 2026 Simultaneous Speech Translation task — The system combines Parakeet and Qwen 3.5 with adaptive policies and RAG for improved SimulST quality and latency.
• Multilingual Reasoning Cascades Need More Context — Multilingual reasoning cascades improve by preserving original query context for final translation.
• Closing the Quality Gap in Low-Resource Text-to-Speech: LoRA Fine-Tuning of VoxCPM2 for Khmer and Korean — LoRA fine-tuning significantly improves low-resource TTS quality, but effectiveness depends on base model weakness.
• Streaming Speech-to-Text Translation with a SpeechLLM — A novel SpeechLLM adaptively decides when to emit translation tokens, achieving low latency and high quality.
• Towards Truly Multilingual ASR: Generalizing Code-Switching ASR to Unseen Language Pairs — Code-switching ASR generalization to unseen language pairs remains limited despite model merging and domain generalization efforts.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.