From Sinhala to Dhivehi: Cross-Lingual Transfer Learning for Low-Resource Speech Recognition

2026-07-08 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Speech Technology · Depth: Expert, extended

Summary

A study investigated cross-lingual transfer learning from Sinhala to improve Automatic Speech Recognition (ASR) for Dhivehi, a low-resource language with only 37 validated hours of speech data. Researchers conducted seventeen experiments across five paradigms: Dhivehi-only baselines, sequential fine-tuning, multilingual fine-tuning, continual pre-training, and a Turkish control. The strongest system, using continual pre-training on Sinhala followed by fine-tuning on Dhivehi with KenLM, achieved a 12.89% Word Error Rate (WER) and 2.70% Character Error Rate (CER). This significantly outperformed the Dhivehi-only baseline of 13.50% WER and 3.02% CER. The research confirmed that linguistic relatedness between Sinhala and Dhivehi, both Insular Indo-Aryan languages, is crucial for transfer gains, as the unrelated Turkish control performed worse. Critically, the external KenLM language model provided the most substantial performance improvement, reducing WER by 24-29 absolute points across all configurations.

Key takeaway

For Machine Learning Engineers developing ASR systems for low-resource languages, you should prioritize continual pre-training from a linguistically related source language. Your implementation must integrate a robust external language model like KenLM, as it provides the most significant performance gains. Be cautious with language ID tokens in bilingual fine-tuning, as they may degrade performance rather than improve it. This approach can yield substantial improvements over monolingual baselines, even surpassing existing benchmarks.

Key insights

Cross-lingual transfer for low-resource ASR benefits from linguistic relatedness and specific adaptation strategies, with external language models being critical.

Principles

• Linguistic relatedness enables effective cross-lingual ASR transfer.
• Continual pre-training adapts representations better than sequential fine-tuning.
• External language models are essential for low-resource ASR performance.

Method

The study used Wav2Vec for ASR, applying sequential fine-tuning, multilingual fine-tuning, and continual pre-training from Sinhala to Dhivehi, with KenLM for decoding.

In practice

• Prioritize continual pre-training for related language transfer.
• Integrate KenLM or similar N-gram LMs for decoding.
• Empirically validate language ID token utility in bilingual setups.

Topics

• Automatic Speech Recognition
• Cross-lingual Transfer Learning
• Low-resource Languages
• Wav2Vec
• KenLM
• Dhivehi Language
• Sinhala Language

Code references

• lukmalilyas/From-Sinhala-to-Dhivehi-ASR

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Low-resource Language Discrimination Towards Chinese Dialects with Transfer learning and Data Augmentation — Transfer learning and data augmentation effectively overcome low-resource challenges in Chinese dialect discrimination.
• Zero-Shot to Full-Resource: Cross-lingual Transfer Strategies for Aspect-Based Sentiment Analysis — Cross-lingual transfer strategies for ABSA vary in effectiveness based on model architecture and resource availability.
• Giving Voice to the Constitution: Low-Resource Text-to-Speech for Quechua and Spanish Using a Bilingual Legal Corpus — Cross-lingual transfer in TTS effectively mitigates data scarcity for low-resource languages like Quechua.
• Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian — QLoRA fine-tuning significantly boosts cross-lingual relation extraction performance for low-resource languages, outperforming prompt-only LLMs.
• The Tatoxa System for Text Detoxification in Low-Resource Languages: The Case of Tatar — The Tatoxa system achieves state-of-the-art text detoxification for Tatar, emphasizing native data's superiority over cross-lingual transfer.
• Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning — Multilingual code-switching instruction tuning improves LLM performance beyond bilingual settings.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.