NAVER LABS System Re-implementation for the IWSLT 2026 Instruction-Following Task

2026-07-06 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing & Speech · Depth: Expert, quick

Summary

NAVER LABS re-implemented its IWSLT 2025 instruction-following pipeline for the IWSLT 2026 Shared Task, specifically for the constrained condition and short audio track. This adaptation integrated mandated components: SeamlessM4T-v2-large as the speech encoder and Qwen3-4B-Instruct as the LLM backbone. The core three-stage approach, involving projector alignment, text-only LoRA pre-training, and multimodal merging, was retained from the original design. Additionally, the team generated 100,000 synthetic instruction-following examples, with 10,000 examples across ten distinct speech-centric task types, using provided corpora for further Stage 3 fine-tuning. The re-implemented primary model achieved a COMET score of 0.781 on EN-ZH speech translation and a BERTScore-F1 of 0.346 on English SQA when evaluated against the MCIF benchmark.

Key takeaway

For NLP Engineers developing multimodal instruction-following systems, this re-implementation demonstrates a successful adaptation strategy. You should consider a three-stage pipeline for integrating new speech encoders and LLM backbones, such as SeamlessM4T-v2-large and Qwen3-4B-Instruct. Generating 100,000 synthetic, task-specific examples can significantly boost Stage 3 fine-tuning performance. This offers a clear path to improve system robustness and task adherence in constrained environments.

Key insights

The re-implementation adapts a three-stage instruction-following pipeline with new mandated components and synthetic data for IWSLT 2026.

Principles

• Three-stage pipeline for instruction following.
• Synthetic data generation enhances fine-tuning.
• Component adaptation maintains core architecture.

Method

The method involves projector alignment, text-only LoRA pre-training, and multimodal merging. It also includes generating 100k synthetic instruction-following examples across ten speech-centric task types for Stage 3 fine-tuning.

In practice

• Use SeamlessM4T-v2-large for speech encoding.
• Integrate Qwen3-4B-Instruct as LLM backbone.
• Generate synthetic data for task-specific fine-tuning.

Topics

• Instruction Following
• Multimodal AI
• Speech Translation
• Large Language Models
• Synthetic Data Generation
• Model Adaptation

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Building AdvancedIF: Evolving Instruction Following Beyond IFEval and “Avoid the Letter C” — Current instruction-following benchmarks often prioritize programmatic verifiability over real-world utility, hindering AI development.
• AlignAtt4LLM: Fast AlignAtt for Decoder-Only LLMs at IWSLT 2026 Simultaneous Speech Translation Task — Adapting AlignAtt for decoder-only LLMs enables simultaneous speech translation by re-engineering attention mechanisms.
• MLLP-VRAIN UPV system for the IWSLT 2026 Simultaneous Speech Translation task — The system combines Parakeet and Qwen 3.5 with adaptive policies and RAG for improved SimulST quality and latency.
• Paper Digest: ICLR 2026 Papers & Highlights — The ICLR 2026 highlights reveal a strong focus on enhancing LLM reasoning, multi-modal capabilities, and robust AI agent development.
• FBK's Long-form SpeechLLMs for IWSLT 2026 Instruction Following — Fixed 30-second segmentation provides robust long-form SpeechLLM performance, mitigating hallucination effects on ASR and SSUM.
• Closing the Gap at CRAC 2026: Two-Stage Adaptation for LLM-Based Multilingual Coreference Resolution — Two-stage fine-tuning and optimized input formats enable LLMs to achieve state-of-the-art multilingual coreference resolution.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.