The CUHKSZ System for the IWSLT 2026 Low-Resource Speech-to-Text Task

2026-06-29 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, short

Summary

The CUHKSZ system, presented at IWSLT 2026, addresses the Low-Resource Speech-to-Text task by introducing Gradient-Driven Parameter Sharing (GDPS). This framework analyzes inter-language gradient behaviors to automatically determine optimal language groupings and shared-private parameter ratios. Built upon SeamlessM4T-Medium, GDPS reduces negative transfer by specializing Layer 11 FFN2 while maintaining shared encoder representations across languages. The system also incorporates curriculum distillation with progressive pseudo-label mixing and test-time reranking, combining prior-BLEU weighting and self-consistency scoring. Evaluation on eight low-resource languages (bem, ckb, gle, hau, ibo, yor, aeb, est) showed significant gains, with bem achieving +2.07 BLEU, hau +1.50, and ibo +0.38 compared to unified fine-tuning. Languages like ckb and yor benefited more from prior-based reranking during inference.

Key takeaway

For NLP Engineers developing multilingual speech-to-text systems, consider integrating Gradient-Driven Parameter Sharing (GDPS) to improve performance on low-resource languages. This approach, which dynamically adjusts parameter sharing and specializes specific layers like FFN2, can significantly reduce negative transfer. You should also explore curriculum distillation and test-time reranking with prior-BLEU weighting to further boost accuracy, especially for languages like ckb and yor.

Key insights

GDPS optimizes low-resource speech-to-text by dynamically sharing parameters based on inter-language gradient analysis.

Principles

• Gradient analysis guides parameter sharing decisions.
• Specialize specific layers to reduce negative transfer.
• Combine curriculum distillation with reranking.

Method

Gradient-Driven Parameter Sharing (GDPS) analyzes inter-language gradients to set language groupings and shared-private parameter ratios. It specializes Layer 11 FFN2 on SeamlessM4T-Medium, integrates curriculum distillation, and uses test-time reranking.

In practice

• Apply GDPS to multilingual speech models.
• Specialize FFN2 for low-resource tasks.
• Use prior-BLEU reranking for inference.

Topics

• Low-Resource ASR
• Gradient-Driven Parameter Sharing
• Multilingual Speech Translation
• SeamlessM4T
• Curriculum Distillation
• Test-Time Reranking

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Giving Voice to the Constitution: Low-Resource Text-to-Speech for Quechua and Spanish Using a Bilingual Legal Corpus — Cross-lingual transfer mitigates data scarcity in low-resource languages for high-quality TTS.
• The Professor: Multi-Teacher Unsupervised Prompt Distillation for Vision-Language Models — Multi-teacher prompt distillation significantly improves VLM student performance, particularly under domain shift, through complementary teacher supervision.
• Low-resource Language Discrimination Towards Chinese Dialects with Transfer learning and Data Augmentation — Transfer learning and data augmentation effectively overcome low-resource challenges in Chinese dialect discrimination.
• NaturalFlow: Reducing Disruptive Pauses for Natural Speech Flow in Simultaneous Speech-to-Speech Translation — Simultaneous S2ST can achieve natural speech flow by optimizing for fluency and translation quality concurrently, avoiding disruptive pauses.
• Source-Grounded Semantic Reinforcement Learning for Low-Resource Target-Language Generation — SG-SRL uses source-language monolingual data with cross-lingual semantic RL to improve low-resource target-language generation, recovering fluency with a small parallel corpus.
• Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language — A compact 135M parameter LLM for Bangla achieves 1B-tier performance by efficiently merging tokens, overcoming low-resource script challenges.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.