SAAKTH at BEA 2026 Shared Task 1: L1-Aware English Vocabulary Difficulty Prediction with Hybrid Transformer and Psycholinguistic Features

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Team SAAKTH's system, presented at the BEA 2026 Shared Task on Vocabulary Difficulty Prediction (Closed Track), addresses the challenge that English word difficulty is not static but varies based on learners' native language (L1). Their approach integrates a fine-tuned XLM-RoBERTa-large encoder with handcrafted psycholinguistic features, which are specifically engineered and optimized for each L1 group. These features are combined using a shallow multilayer perceptron. The system ensures stability through five-seed ensembling and XGBoost-based blending. On the development set, the system achieved Root Mean Square Errors (RMSEs) of 0.997 for Spanish (es), 1.002 for German (de), and 0.932 for Chinese (cn) learners. These results represent a significant 20–25% improvement over the baseline, underscoring the effectiveness of L1-aware modeling, particularly in scenarios with limited data.

Key takeaway

For NLP Engineers developing vocabulary assessment tools for English language learners, you should integrate L1-aware modeling into your system designs. Recognizing that word difficulty varies by native language allows for more accurate predictions, as demonstrated by 20-25% improvements over baselines. Consider combining large language models with L1-specific psycholinguistic features and employing ensembling techniques to enhance model stability and performance, particularly when data is scarce.

Key insights

L1-aware modeling significantly improves English vocabulary difficulty prediction, especially with limited data.

Principles

• Word difficulty is L1-dependent.
• Hybrid models outperform single-feature approaches.
• Ensembling enhances model stability.

Method

Combines XLM-RoBERTa-large with L1-specific psycholinguistic features via MLP, optimized per L1, then blended with XGBoost and five-seed ensembling.

In practice

• Engineer features tailored to L1 groups.
• Use ensembling for robust predictions.
• Integrate psycholinguistic data into NLP.

Topics

• Vocabulary Difficulty Prediction
• L1-Aware Modeling
• XLM-RoBERTa
• Psycholinguistic Features
• Ensemble Learning
• Educational NLP

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction — Soft-target loss fine-tuning significantly improves LLM and MLM performance in continuous value prediction tasks.
• Global vs. Local Sentence Embeddings for Brazilian Portuguese: Revisiting Monolingual Models in the Age of Foundation Models — Optimal model choice for PT-BR sentence embeddings balances vocabulary coverage and adaptation flexibility.
• Towards interpretable models for language proficiency assessment: Predicting the CEFR level of Estonian learner texts — Careful feature selection yields interpretable, generalizable models for language proficiency assessment with high accuracy.
• Linguistic Issues in Computational Linguistics for Non-English Languages — English-centric LLM design creates systematic linguistic weaknesses for non-English languages across multiple levels.
• What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction — LLM fine-tuning with soft-target loss effectively predicts vocabulary difficulty while explainable models offer insights.
• As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language — LLMs struggle with combined negation and figurative language, with prompt style significantly impacting performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.