Data-Efficient Adaptation of Multilingual LLMs to Ukrainian

2026-06-28 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

A reproducible approach addresses challenges in adapting large language models to low-resource languages, focusing on inefficient tokenization, data scarcity, and limited instruction tuning resources. This method combines vocabulary surgery for tokenizer adaptation without full retraining, cross-lingual transfer of quality classifiers via translation for data filtering, and instruction data generation through translation, task conversion, and targeted synthesis. Validated by adapting Gemma-3-12B to Ukrainian, the pretrained model achieves top performance on Ukrainian benchmarks. Its instruction-tuned variant demonstrates strong performance, including 33 BLEU on FLORES for translation, and excels in summarization and question-answering tasks. Notably, it requires 1.5x fewer tokens than the original model for equivalent text processing. All models, datasets, classifiers, and code are released for replication.

Key takeaway

For NLP Engineers adapting large language models to low-resource languages, this data-centric approach provides a validated blueprint to overcome common challenges. You can achieve strong performance on tasks like translation and summarization while significantly reducing token requirements. Consider implementing vocabulary surgery, cross-lingual quality classifier transfer, and instruction data synthesis to efficiently deploy models like Gemma-3-12B in new linguistic contexts.

Key insights

Data-centric methods enable efficient adaptation of multilingual LLMs to low-resource languages by addressing tokenization, data scarcity, and instruction tuning.

Principles

• Vocabulary surgery adapts tokenizers without full retraining.
• Cross-lingual transfer enables data filtering without target annotations.
• Instruction data can be synthesized via translation and task conversion.

Method

The approach involves vocabulary surgery, cross-lingual quality classifier transfer via translation, and instruction data generation through translation, task conversion, and targeted synthesis.

In practice

• Adapt Gemma-3-12B for low-resource languages.
• Achieve 33 BLEU on FLORES for translation tasks.
• Reduce token count by 1.5x for text processing.

Topics

• Multilingual LLMs
• Low-Resource Languages
• Data-Efficient Adaptation
• Tokenizer Adaptation
• Instruction Tuning
• Gemma-3-12B

Best for: Research Scientist, AI Engineer, Machine Learning Engineer, NLP Engineer, AI Scientist

Related on AIssential

• Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning — Multilingual code-switching instruction tuning improves LLM performance beyond bilingual settings.
• Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language — A compact 135M parameter LLM for Bangla achieves 1B-tier performance by efficiently merging tokens, overcoming low-resource script challenges.
• Mining Useful General Data for Low-Resource Domain Adaptation — Neural Tangent Kernels can effectively select high-value auxiliary data for low-resource LLM domain adaptation.
• Linguistic Issues in Computational Linguistics for Non-English Languages — English-centric LLM design creates systematic linguistic weaknesses for non-English languages across multiple levels.
• A NATO-accredited organization published a report concerning language-based disparities in LLM performance (Published April 25, 2026) — LLM performance varies significantly by language, but adaptation strategies can bridge the gap.
• System Report for CCL25-Eval Task 5: New Dataset and LoRA-Fine-Tuned Qwen2.5 — Domain-specific LoRA fine-tuning on Qwen2.5-14B with CCPoetry-49K significantly enhances classical Chinese poetry understanding and translation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.