The FBK Sentence-Aware Subtitling System at the IWSLT 2026 Subtitling Track

2026-06-29 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Speech & Language Processing · Depth: Expert, quick

Summary

The FBK Sentence-Aware Subtitling System was submitted to the IWSLT 2026 Subtitling track, which required automatically subtitling English audio-visual content. This task covered three domains: ITV entertainment, Asharq-Bloomberg news, and YouTube YODAS dataset recordings, targeting up to four languages from a pool of five (Arabic, Chinese, German, Japanese, and Spanish). The system uses an ASR-MT cascade framework built entirely from freely available open-source components. Its primary two-stage pipeline first produces time-aligned subtitles via voice activity detection, automatic transcription, and subtitle-level translation. A second refinement stage then re-processes audio with long-form transcription and sentence-level translation, re-aligning output to original timings to improve quality while preserving synchronization. Two contrastive systems were also submitted.

Key takeaway

For NLP Engineers designing or evaluating subtitling pipelines, this system demonstrates that a two-stage ASR-MT cascade can significantly enhance transcription and translation quality. By initially generating time-aligned subtitles and then refining them with broader context and sentence-level translation, you can achieve superior results without sacrificing crucial synchronization. Consider implementing similar multi-stage architectures, especially when dealing with diverse audio-visual content and strict timing requirements.

Key insights

The FBK system uses a two-stage ASR-MT cascade with context-aware refinement to improve subtitling quality while preserving synchronization.

Principles

• Utilize open-source components.
• Preserve synchronization constraints.
• Leverage broader context for quality.

Method

A two-stage pipeline: initial time-aligned subtitles (VAD, ASR, subtitle MT), followed by refinement using long-form ASR and sentence-level MT, then re-alignment to original timings.

In practice

• Subtitle English audio-visual content.
• Translate across diverse domains.
• Support multiple target languages.

Topics

• Subtitling Systems
• Automatic Speech Recognition
• Machine Translation
• IWSLT Evaluation
• Open-source AI
• Audio-visual Processing

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Streaming Speech-to-Text Translation with a SpeechLLM — A novel SpeechLLM adaptively decides when to emit translation tokens, achieving low latency and high quality.
• MTR-DuplexBench: Towards a Comprehensive Evaluation of Multi-Round Conversations for Full-Duplex Speech Language Models — MTR-DuplexBench offers a comprehensive, multi-round evaluation for Full-Duplex Speech Language Models.
• Multilingual Reasoning Cascades Need More Context — Multilingual reasoning cascades improve by preserving original query context for final translation.
• SPRI: SVD-Partitioned Residual Initialization for Data-Constrained MoE Upcycling — SPRI leverages SVD-partitioned residuals from pretrained FFNs to create diverse MoE experts, improving upcycling under data constraints.
• A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech Translation — A new evaluation method for long-form SimulS2ST uses ASR, forced alignment, and sentence embeddings to measure latency and quality.
• How Vimeo Implemented AI-Powered Subtitles — LLMs optimized for fluency can break structural constraints, requiring architectural separation of concerns.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.