HydraQE: OSU’s Submission for the IWSLT 2026 Speech Translation Metrics Shared Task

2026-06-29 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

HydraQE, Ohio State University's submission for the IWSLT 2026 Speech Translation Metrics shared task, is an end-to-end, reference-free quality estimation (QE) system. It uses a Qwen3-ASR backbone, accepting source audio and a translation hypothesis as joint input. The system combines hidden states from all backbone layers via a sparsemax scalar mix, then re-encodes them with a bidirectional Transformer for cross-modal interaction. To address data scarcity, HydraQE trains three independent prediction heads on human direct assessment (DA) annotations, MetricX-24 pseudo-labels, and xCOMET pseudo-labels. It employs a curriculum that starts with synthetic and silver pseudo-labeled data, gradually shifting to human-annotated examples. HydraQE outperforms cascaded text-based baselines and prior direct speech QE systems, proving end-to-end speech translation QE is competitive.

Key takeaway

For NLP Engineers developing speech translation quality estimation systems, HydraQE demonstrates that end-to-end approaches can outperform traditional cascaded text-based baselines. You should consider integrating multi-modal inputs and diverse supervision signals, including pseudo-labels like MetricX-24 and xCOMET, to overcome data scarcity. Employing a curriculum learning strategy, starting with synthetic data and progressing to human annotations, can significantly enhance your system's performance and robustness.

Key insights

End-to-end speech translation QE can achieve competitive performance by integrating multi-modal inputs and diverse supervision signals.

Principles

• Combine all backbone layer hidden states via sparsemax scalar mix.
• Re-encode for full cross-modal interaction using a bidirectional Transformer.
• Train on complementary supervision signals to mitigate data scarcity.

Method

HydraQE uses a Qwen3-ASR backbone, combines hidden states via sparsemax, re-encodes with a bidirectional Transformer, and trains three prediction heads on human DA, MetricX-24, and xCOMET pseudo-labels using a curriculum.

In practice

• Utilize Qwen3-ASR as a robust ASR backbone for QE.
• Integrate MetricX-24 and xCOMET pseudo-labels for training.
• Implement curriculum learning for scarce human-annotated data.

Topics

• Speech Translation
• Quality Estimation
• IWSLT 2026
• Qwen3-ASR
• Multi-modal AI
• Curriculum Learning
• Pseudo-labeling

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech Translation — A new evaluation method for long-form SimulS2ST uses ASR, forced alignment, and sentence embeddings to measure latency and quality.
• Introducing Post-Stream Refinement: Higher-Accuracy Real-Time Transcription — Post-Stream Refinement enhances real-time speech recognition by combining instant partial results with a higher-accuracy, context-aware second pass.
• XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation Metrics — Cross-lingual scoring bias in translation metrics can be identified and mitigated using controlled, parallel-quality datasets.
• MLLP-VRAIN UPV system for the IWSLT 2026 Simultaneous Speech Translation task — The system combines Parakeet and Qwen 3.5 with adaptive policies and RAG for improved SimulST quality and latency.
• Diffusion Language Models for Speech Recognition — Diffusion language models significantly improve speech recognition accuracy through novel rescoring and joint-decoding methods.
• Improving quality and robustness in LLM-based text-to-speech systems — LLM-based TTS quality improves through targeted techniques addressing accent leakage, expressiveness, and reliability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.