Data Asgardians at BEA 2026 Shared Task 1: A Hybrid Transformer–Feature Ensemble for L1-Aware English Vocabulary Difficulty Prediction

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

The "Data Asgardians" team developed a hybrid system for the BEA 2026 Shared Task 1, focusing on L1-aware English vocabulary difficulty prediction. This system predicts psychometrically calibrated GLMM difficulty scores for English vocabulary items, considering learners' first-language backgrounds: Spanish (ES), German (DE), and Mandarin Chinese (CN). Their approach integrates 33 hand-crafted linguistic features, including phonological, morphological, semantic, contextual, and cross-lingual aspects, with contextual multilingual transformer representations. Initial Closed Track submissions utilized XLM-RoBERTa-large Solo and Hybrid models, achieving test RMSEs of 1.182 (ES), 1.117 (DE), and 1.006 (CN), with a mean of 1.103, surpassing the official baseline. A subsequent refinement using mDeBERTa-v3-base components and a Ridge stacking ensemble further decreased the mean test RMSE to 0.982, with individual scores of 1.037 (ES), 0.997 (DE), and 0.913 (CN), marking a 0.121 improvement.

Key takeaway

For NLP Engineers developing language learning applications, integrating both hand-crafted linguistic features and multilingual transformer models is crucial for accurate vocabulary difficulty prediction. You should consider L1 backgrounds as a key variable and explore ensemble techniques like Ridge stacking to achieve lower RMSEs, as demonstrated by the 0.982 mean test RMSE. This hybrid approach significantly outperforms transformer-only or feature-only systems, enhancing the precision of educational tools.

Key insights

Combining hand-crafted linguistic features with multilingual transformers significantly improves L1-aware vocabulary difficulty prediction.

Principles

• Linguistic features complement transformer representations.
• Ensembling models enhances prediction accuracy.
• L1 background is crucial for vocabulary difficulty.

Method

The method involves engineering 33 linguistic features, combining them with multilingual transformer models (XLM-RoBERTa-large, mDeBERTa-v3-base), and employing prediction-level ensembling, specifically a Ridge stacking ensemble, to predict GLMM difficulty scores.

In practice

• Integrate L1-specific features in NLP tasks.
• Use hybrid models for robust predictions.
• Apply ensemble methods to reduce RMSE.

Topics

• Vocabulary Difficulty Prediction
• Multilingual Transformers
• Linguistic Features
• Ensemble Learning
• L1-Aware NLP
• Educational Applications

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction — Soft-target loss fine-tuning significantly improves LLM and MLM performance in continuous value prediction tasks.
• Automatic Evaluation of Enem Essays: A Comparative Analysis between Feature Engineering and Transformers — Regression formulations are generally more suitable for ordinal essay scoring than multiclass classification.
• An End-to-End Hybrid Framework for Rumour Detection in Low-Resources Algerian Dialect — A hybrid framework combining transformer embeddings with classical classifiers enables effective rumour detection in low-resource Algerian dialect.
• Linguistic Issues in Computational Linguistics for Non-English Languages — English-centric LLM design creates systematic linguistic weaknesses for non-English languages across multiple levels.
• Towards interpretable models for language proficiency assessment: Predicting the CEFR level of Estonian learner texts — Careful feature selection yields interpretable, generalizable models for language proficiency assessment with high accuracy.
• State-of-the-Art Transformer Pipelines in spaCy — spaCy offers robust, configurable Transformer pipelines for NLP tasks, often outperforming LLMs on predictive tasks with modest data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.