Echoes Across Vietnam's Highlands, Delta, and Coast: A Multilingual Corpus for Cham, Khmer, and Tay-Nung

2026-07-20 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, extended

Summary

CKTN, the first multilingual corpus and benchmark for Vietnam's Cham, Khmer, and Tay-Nung languages, addresses their severe underrepresentation in NLP. Comprising 44,367 documents and 24M subword tokens, CKTN supports continued pretraining, category classification, and summary-document retrieval. The research reveals that existing multilingual encoders like mBERT, XLM-R, and RemBERT heavily fragment these languages, and standard metrics such as MLM loss and lexical-overlap retrieval can misrepresent model quality. To counter this, a novel script-aware adaptation recipe, combining vocabulary augmentation with calibrated replaced-token pretraining, was developed. This approach prevents discriminators from exploiting superficial script mismatches, yielding CKTN-ELECTRA, an encoder with significantly reduced fragmentation and superior classification performance (0.9214 accuracy, 0.7103 Macro-F1) compared to other models.

Key takeaway

For NLP engineers adapting multilingual models to low-resource, script-diverse languages, you should prioritize vocabulary augmentation and calibrated replaced-token pretraining. Your evaluation should emphasize classification performance over lexical-overlap retrieval, as the latter can mask poor semantic generalization. Implement script-aware filtering and linear loss scheduling in ELECTRA-style pretraining to avoid trivial shortcut learning and achieve robust cross-document semantic understanding.

Key insights

Effective NLP for script-diverse, low-resource languages demands tailored adaptation methods that overcome tokenizer fragmentation and misleading evaluation metrics.

Principles

• Multilingual encoders disproportionately fragment low-resource languages.
• MLM loss and lexical-overlap retrieval can misrepresent semantic generalization.
• Script heterogeneity poses a specific risk for ELECTRA-style pretraining.

Method

The adaptation recipe combines vocabulary augmentation (candidate selection, score calibration, embedding initialization) with script-aware calibrated replaced-token pretraining, using a smaller generator, difficulty-calibrated replacement sampling, and linear loss scheduling.

In practice

• Augment vocabulary to reduce fragmentation in low-resource language models.
• Implement script-aware filtering for replaced tokens to prevent shortcut learning.
• Use linear scheduling to stabilize generator-discriminator training in RTD.

Topics

• Low-Resource NLP
• Multilingual Language Models
• Vocabulary Augmentation
• ELECTRA Pretraining
• Cham Language
• Khmer Language
• Tay-Nung Language

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Echoes Across Vietnam's Highlands, Delta, and Coast: A Multilingual Corpus for Cham, Khmer, and Tay-Nung — CKTN corpus and script-aware adaptation significantly improve NLP for underrepresented Vietnamese minority languages, revealing flaws in standard evaluation metrics.
• PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation — Ensembling LoRA-tuned Gemma models with LLM-generated synthetic data and per-language thresholding significantly improves multilingual polarization detection generalization.
• Low-Resource Methods for Hawaiian Machine Translation — Multilingual training and simpler models like mBART are effective for low-resource machine translation of endangered languages.
• Closing the Quality Gap in Low-Resource Text-to-Speech: LoRA Fine-Tuning of VoxCPM2 for Khmer and Korean — LoRA fine-tuning significantly improves low-resource TTS quality, but effectiveness depends on base model weakness.
• Low-resource Language Discrimination Towards Chinese Dialects with Transfer learning and Data Augmentation — Transfer learning and data augmentation effectively overcome low-resource challenges in Chinese dialect discrimination.
• Scripts Through Time: A Survey of the Evolving Role of Transliteration in NLP — Transliteration bridges the "script barrier" in NLP, enhancing cross-lingual transfer and efficiency, especially for low-resource languages.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.