CUHKSZ Simultaneous Speech Translation System for IWSLT 2026

2026-06-29 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

The CUHKSZ Team submitted their system to the IWSLT 2026 Simultaneous Speech Translation evaluation, focusing on English→Chinese and English→German for unsegmented speech. Their architecture is based on Qwen3-Omni-30B-A3B, a natively aligned audio-text LLM, with LoRA adaptation applied under constrained conditions. They generated syntax-aware, chunk-aligned supervision data using Qwen3-30B-Instruct to synthesize target translations, enabling the model to internalize a simultaneous read/write policy. Execution is handled by a lightweight streaming agent via vLLM, managing fixed audio chunks, dialogue history, and emission controls. The system achieved 40.5 BLEU (1.95 s) for En→Zh and 27.7 BLEU (1.72 s) for En→De in the 0–2 s latency regime, improving to 42.1 BLEU (2.16 s) and 30.5 BLEU (2.29 s) in the 2–4 s regime.

Key takeaway

For Machine Learning Engineers developing real-time speech translation, this system offers a robust blueprint. You should consider integrating natively aligned audio-text LLMs like Qwen3-Omni-30B-A3B and implementing a policy internalization approach. This method, combined with a vLLM-based streaming agent, can achieve competitive BLEU scores at sub-2-second latencies, significantly enhancing user experience in live translation scenarios.

Key insights

A simultaneous speech translation system internalizes read/write policies for low-latency, high-BLEU performance using an audio-text LLM.

Principles

• LoRA adaptation can be applied exclusively to LLMs under constrained conditions.
• Synthesizing target translations with an instruction model creates supervision data.
• Internalizing read/write policies enables autonomous token prediction at semantic boundaries.

Method

Construct syntax-aware, chunk-aligned supervision from ASR corpora using Qwen3-30B-Instruct to synthesize translations, then adapt Qwen3-Omni-30B-A3B via LoRA for policy internalization.

In practice

• Utilize Qwen3-Omni-30B-A3B for audio-text LLM simultaneous translation.
• Deploy a vLLM-served streaming agent with strict emission controls.
• Dynamically inject contextual priors into prompts for sub-tracks.

Topics

• Simultaneous Speech Translation
• Large Language Models
• LoRA Adaptation
• Qwen3-Omni-30B-A3B
• Low-Latency Inference
• vLLM

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Streaming Speech-to-Text Translation with a SpeechLLM — A novel SpeechLLM adaptively decides when to emit translation tokens, achieving low latency and high quality.
• MLLP-VRAIN UPV system for the IWSLT 2026 Simultaneous Speech Translation task — The system combines Parakeet and Qwen 3.5 with adaptive policies and RAG for improved SimulST quality and latency.
• System Report for CCL25-Eval Task 5: New Dataset and LoRA-Fine-Tuned Qwen2.5 — Domain-specific LoRA fine-tuning on Qwen2.5-14B with CCPoetry-49K significantly enhances classical Chinese poetry understanding and translation.
• Grandes Modelos de Linguagem Multimodais (MLLMs): Da Teoria \`a Pr\'atica — MLLMs merge LLM language capabilities with multimodal perception, advancing AI's understanding of diverse data.
• Kyutai Releases Hibiki-Zero: A3B Parameter Simultaneous Speech-to-Speech Translation Model Using GRPO Reinforcement Learning Without Any Word-Level Aligned Data — Hibiki-Zero enables simultaneous speech translation without word-level alignments using GRPO reinforcement learning.
• Alibaba Qwen Team Introduces Qwen3.5-LiveTranslate-Flash: Real-Time Multimodal Interpretation Across 60 Languages at 2.8-Second Latency — Alibaba's Qwen3.5-LiveTranslate-Flash delivers real-time, multimodal interpretation across 60 languages with 2.8-second latency and voice cloning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.