QomL’aqtaqa: A Qom–Spanish Parallel Corpus for Natural Language Processing with Machine Translation Evaluation
Summary
QomL'aqtaqa is the first computationally usable Qom–Spanish parallel corpus, developed for natural language processing and speech processing of Qom, a low-resource Guaycuruan language. It comprises 33,392 parallel segments, including 1,469,905 Qom tokens and 891,344 Spanish tokens, with a 2,943-segment subset excluding Bible-derived content. The corpus features alignments at sentence, fragment, and paragraph levels, compiled from various sources. Researchers established bidirectional neural machine translation baselines using NLLB-200, achieving competitive performance on the full dataset but lower results on the non-Bible subset. An ablation study confirmed that training exclusively on biblical data diminishes performance on non-biblical text, underscoring the necessity of domain diversity in low-resource machine translation.
Key takeaway
For NLP engineers developing machine translation systems for low-resource languages, you should prioritize creating diverse training corpora beyond single-domain sources like religious texts. Evaluating your models on non-domain-specific subsets is critical to ensure generalizability, as relying solely on homogeneous data can significantly degrade performance on varied content. Actively seek out and integrate varied textual sources.
Key insights
Domain diversity is critical for effective low-resource neural machine translation performance.
Principles
- Domain diversity improves low-resource NMT.
- Biblical data alone reduces non-biblical text performance.
- Parallel corpora can align at multiple levels.
Method
Bidirectional NMT baselines were established using NLLB-200, evaluating performance on full and non-Bible Qom-Spanish subsets, with an ablation study on data domain impact.
In practice
- Develop diverse corpora for low-resource languages.
- Evaluate NMT on domain-specific subsets.
- Consider NLLB-200 for low-resource NMT baselines.
Topics
- Qom Language
- Low-Resource NLP
- Parallel Corpora
- Neural Machine Translation
- NLLB-200
- Domain Diversity
Best for: Research Scientist, NLP Engineer, Machine Learning Engineer, AI Scientist
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.