NLP-Explorers at BEA 2026 Shared Task 1: DeBERTa–CatBoost Weighted Ensemble Approach for L1-Specific Vocabulary Difficulty Prediction

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

The NLP-Explorers team at BEA 2026 Shared Task 1 developed a DeBERTa–CatBoost weighted ensemble approach for L1-specific vocabulary difficulty prediction. This system estimates continuous difficulty scores for English target words, incorporating learner-specific information. The core of the approach involves a fine-tuned DeBERTa v3 Large model combined with a CatBoost regressor, which is trained on transformer-based embeddings. The final difficulty score is generated through weighted ensembling, where DeBERTa provides the primary prediction and CatBoost contributes a complementary signal. The system demonstrated reliable performance, achieving RMSE scores of 1.040 for Spanish, 0.992 for German, and 0.882 for Chinese. Its results also showed stability across multiple runs, indicating consistent behavior even with minor changes in ensemble weight.

Key takeaway

For NLP Engineers developing educational applications, if you are building systems for personalized vocabulary learning, consider integrating a hybrid model approach. Combining powerful contextual models like DeBERTa with lightweight regressors such as CatBoost, through weighted ensembling, can yield stable and accurate learner-specific word difficulty predictions. This strategy allows you to effectively model the nuanced, context-dependent nature of vocabulary acquisition.

Key insights

Combining strong contextual representations with a lightweight regression model effectively models learner-sensitive word difficulty.

Principles

• Word difficulty is learner-specific and context-dependent.
• Hybrid systems offer reliable performance.
• Ensembling strong and complementary models improves prediction.

Method

Fine-tune DeBERTa v3 Large, train CatBoost on transformer embeddings, then combine predictions via weighted ensembling for continuous difficulty scores.

In practice

• Use DeBERTa v3 Large for primary contextual embeddings.
• Integrate CatBoost for complementary signal.
• Apply weighted ensembling for robust predictions.

Topics

• Vocabulary Difficulty Prediction
• DeBERTa v3 Large
• CatBoost
• Weighted Ensembling
• Educational NLP
• Learner Modeling

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction — Soft-target loss fine-tuning significantly improves LLM and MLM performance in continuous value prediction tasks.
• JabuticaBERT: Modern Portuguese Encoders from Scratch with RTD and Long-Context Training — Monolingual encoders with careful architecture and tokenization can achieve competitive performance without massive scaling.
• AdaBoost - Explained — AdaBoost combines weak classifiers by adaptively weighting data points and their votes to form a strong ensemble.
• Comparison of Modern Multilingual Text Embedding Techniques for Hate Speech Detection Task — Supervised models with multilingual embeddings and gradient boosted trees excel at hate speech detection.
• Extending an Ensemble Baseline with Corpus-Based Graph Features for Portuguese Pun Detection — Graph-based features can augment pun detection, but their utility depends on specific lexical relation encoding.
• Adapting TrOCR for Printed Tigrinya Text Recognition: Word-Aware Loss Weighting for Cross-Script Transfer Learning — Adapting TrOCR for Tigrinya text requires tokenizer extension and Word-Aware Loss Weighting to overcome script-specific challenges.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.