UGA Threshold at BEA 2026 Shared Task 1: Predicting Vocabulary Acquisition Difficulty with Hand-Crafted SLA-Based Features

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Natural Language Processing · Depth: Advanced, quick

Summary

The UGA Threshold system, submitted to the BEA 2026 Shared Task 1, predicts English vocabulary acquisition difficulty for learners. This feature-based system integrates linguistically motivated features, including frequency, cross-linguistic similarity, phonological and orthographic complexity, and semantic properties. It also incorporates multilingual embeddings, reduced using PCA. Multiple regression models, evaluated via cross-validation, generated final predictions from ensemble and single-model configurations per language. The system achieved competitive performance across German, Spanish, and Chinese L1 groups, surpassing the XLM-RoBERTa baseline in seven of nine RMSE runs. Strongest improvements were noted for Chinese learners, with modest gains for Spanish. An ablation study confirmed that frequency and cross-linguistic similarity are the most substantial contributors to predictive performance, with their effects varying across different L1s.

Key takeaway

For NLP Engineers or AI Scientists developing language learning tools, you should prioritize integrating interpretable, linguistically motivated features like word frequency and cross-linguistic similarity. This approach can yield superior performance in predicting vocabulary acquisition difficulty, especially when tailored to specific L1 learner groups. Consider how L1-specific feature weighting could further refine your models and improve learner outcomes.

Key insights

Linguistically motivated features effectively predict vocabulary difficulty, outperforming baseline models.

Principles

• Interpretable linguistic features enhance vocabulary difficulty prediction.
• Feature importance varies significantly across different L1 groups.
• Cross-linguistic similarity is a key predictor of acquisition difficulty.

Method

The method combines hand-crafted linguistic features (frequency, cross-linguistic similarity, phonological/orthographic complexity, semantic properties) with PCA-reduced multilingual embeddings, training multiple regression models for prediction.

In practice

• Prioritize frequency and cross-linguistic similarity features.
• Tailor feature sets to specific L1 learner populations.
• Consider ensemble models for robust difficulty predictions.

Topics

• Vocabulary Acquisition
• Second Language Acquisition
• Linguistic Features
• Machine Learning Models
• BEA Shared Task
• English Language Learning

Best for: AI Scientist, Research Scientist, NLP Engineer

Related on AIssential

• What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction — LLM fine-tuning with soft-target loss effectively predicts vocabulary difficulty while explainable models offer insights.
• Towards interpretable models for language proficiency assessment: Predicting the CEFR level of Estonian learner texts — Careful feature selection yields interpretable, generalizable models for language proficiency assessment with high accuracy.
• Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor Processing — LLMs struggle with deep metaphor comprehension, often misinterpreting concepts and relying on lexical associations over context or full syntactic understanding.
• What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction — Soft-target loss fine-tuning significantly improves LLM and MLM performance in continuous value prediction tasks.
• Teaching Machines to Read: How We Turn Words Into Numbers — Converting text into numerical vectors is the foundational step for any AI to process human language.
• CLP-Transfer: Cross-Lingual and Progressive Transfer Learning — CLP-Transfer simplifies cross-lingual model transfer using token overlap and a small helper model, but shows limited downstream task performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.