Automated CEFR-Level Assignment for Ukrainian Texts

2026-06-28 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, short

Summary

A study evaluated CEFR-based text complexity for Ukrainian using a new dataset derived from language learner textbooks. Researchers compared traditional machine learning, transformer-based models, and LLM-based evaluation across A1–B2 proficiency levels. A Random Forest classifier achieved the highest macro-F1 score of 0.576, slightly surpassing a fine-tuned XLM-RoBERTa model at 0.574. While GPT-5.5 demonstrated strong performance with a macro-F1 of 0.564, marking an advancement over GPT-4.1, supervised models ultimately achieved marginally better scores in this specific proficiency-level assessment. These findings indicate that structured linguistic analysis offers a robust alternative to purely neural approaches for Ukrainian CEFR classification.

Key takeaway

For NLP engineers developing Ukrainian text complexity tools, this research suggests that focusing on explicit linguistic features with traditional machine learning models like Random Forest can yield competitive results. You should not automatically assume that large language models will provide superior performance for CEFR classification, especially given the Random Forest classifier's macro-F1 of 0.576, which slightly outperformed GPT-5.5's 0.564 in this experiment.

Key insights

Explicit linguistic features are highly effective for automated CEFR-level assignment in Ukrainian texts.

Principles

• Explicit linguistic features remain highly effective for text complexity.
• Supervised ML can outperform LLMs for specific classification tasks.

Method

The study compared traditional ML (Random Forest), transformer-based (XLM-RoBERTa), and LLM (GPT-5.5, GPT-4.1) models for Ukrainian CEFR classification on a new textbook-derived dataset across A1–B2 levels.

In practice

• Consider Random Forest for Ukrainian CEFR classification.
• Prioritize explicit linguistic features in text complexity models.

Topics

• CEFR Classification
• Ukrainian NLP
• Text Complexity
• Machine Learning
• Transformer Models
• Large Language Models
• Random Forest

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Towards interpretable models for language proficiency assessment: Predicting the CEFR level of Estonian learner texts — Careful feature selection yields interpretable, generalizable models for language proficiency assessment with high accuracy.
• NLP-based Page Classification for Efficient LLM Extraction from Brazilian Public Tender Documents — Combining NLP page classification with LLM extraction significantly reduces token count and cost for document processing.
• An End-to-End Hybrid Framework for Rumour Detection in Low-Resources Algerian Dialect — A hybrid framework combining transformer embeddings with classical classifiers enables effective rumour detection in low-resource Algerian dialect.
• Automatic Question classification in Portuguese: A Large-Scale Dataset and Comparative Evaluation of Classification Strategies — A new 17,000-question Portuguese dataset enables large-scale automatic classification of exam questions.
• How Far Can Classical NLP Go? From Bag-of-Words to Stacking on Spooky Author Identification — Classical NLP, with careful feature engineering and ensemble stacking, excels at stylistic text classification.
• [P] QLoRA Fine-Tuning of Qwen2.5-1.5B for CEFR English Proficiency Classification (A1–C2) [P] — QLoRA fine-tuning of Qwen2.5-1.5B effectively classifies CEFR English proficiency, despite synthetic data limitations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.