BSC’s Submission to the Instruction Following Track of IWSLT 2026

2026-06-29 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, medium

Summary

The Barcelona Supercomputing Center (BSC) submitted an end-to-end (E2E) system to the Instruction Following (IF) track of IWSLT 2026. This system integrates a speech encoder with a translation-oriented Large Language Model, designed to handle multiple tasks through natural language instructions. It is trained on both speech and text data, encompassing automatic speech recognition, translation, question answering, and instruction following. A core strategy is Chain-of-Thought (CoT) generation, which explicitly decomposes tasks by first producing an intermediate transcription. This approach enhances the reuse of text-only supervision and boosts robustness. The system also employs diverse prompt formulations and aligns text-only and speech inputs under a shared inference pattern to improve generalization. Evaluated on IWSLT 2025 data, the BSC approach achieved competitive and state-of-the-art performance across various tasks.

Key takeaway

For AI Scientists developing unified spoken language systems, this research highlights the effectiveness of Chain-of-Thought (CoT) generation. You should consider implementing CoT to explicitly decompose tasks, producing intermediate transcriptions to enhance robustness and leverage existing text-only supervision. Aligning diverse prompt formulations and inference patterns for both speech and text inputs can further improve generalization and competitive performance in multi-task instruction following.

Key insights

BSC's E2E system uses CoT generation and aligned inputs for robust, multi-task spoken language instruction following.

Principles

• Decompose complex tasks via intermediate steps.
• Align multi-modal inputs for generalization.
• Reuse text-only data for speech systems.

Method

The system combines a speech encoder with a translation-oriented LLM, employing Chain-of-Thought to generate intermediate transcriptions before final output. Diverse prompts and shared inference patterns support generalization.

In practice

• Implement CoT for spoken language tasks.
• Design prompts for multi-modal instruction following.
• Integrate speech encoders with LLMs.

Topics

• Instruction Following
• Spoken Language Systems
• Chain-of-Thought
• Large Language Models
• Speech Recognition
• Multimodal AI

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Theory of Mind in Action: The Instruction Inference Task in Dynamic Human-Agent Collaboration — Few-shot chain-of-thought prompting significantly enhances LLM Theory of Mind for collaborative task interpretation.
• LEAF: Growing Trees Without Branching for Speech-Aware Large Language Model Post-Training — LEAF is a retrospective tree-based RL method for speech-aware LLM post-training that refines credit assignment.
• IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation — IV-CoT improves text-to-image generation by separating structural planning from appearance rendering using a latent visual reasoning cascade.
• Analyzing Chain of Thought (CoT) Approaches in Control Flow Code Deobfuscation Tasks — CoT prompting significantly improves LLM performance in complex code deobfuscation tasks.
• ACIL: Auto Chain of Thoughts for In-Context Learning — Auto-CoT improves LLM in-context learning by automatically generating, pruning, and selecting reasoning chains.
• Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation — Fine-tuned 8B-parameter language models can objectively forecast research idea success by comparing empirical outcomes, outperforming larger frontier models.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.