WSE Research at BEA 2026 Shared Task 2: Multi-Strategy Rubric-Based Short Answer Scoring for German

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

The WSE Research system, developed for the BEA 2026 Shared Task 2 on Rubric-based Short Answer Scoring for German, integrates multiple strategies. It combines rubric-conditioned prompting, TF-IDF exemplar retrieval, LoRA fine-tuning of open-source Qwen models, and prediction aggregation. On the ALICE-LP-1.0 trial set, a fine-tuned Qwen2.5-32B achieved a QWK score of 0.769, surpassing the Gemini 3 Flash commercial baseline's 0.748. The system secured second place on three tracks and third on one track in the official test set. These results indicate that rubric-conditioned fine-tuning offers a competitive and cost-effective alternative to commercial APIs for German short answer scoring, with aggregation benefiting seen questions, while larger single models generalize better to unseen rubrics.

Key takeaway

For NLP Engineers developing German short answer scoring systems, consider fine-tuning open-source models like Qwen with rubric-conditioned data. Your approach can achieve competitive performance, potentially exceeding commercial APIs, while being more cost-effective. Implement prediction aggregation for improved accuracy on familiar question types, but prioritize larger single models for better generalization to novel rubrics.

Key insights

Fine-tuning open-source models with multi-strategy approaches can outperform commercial APIs for rubric-based short answer scoring.

Principles

• Rubric-conditioned fine-tuning is cost-effective.
• Aggregation aids seen questions.
• Larger models generalize better to unseen rubrics.

Method

The system combines rubric-conditioned prompting, TF-IDF exemplar retrieval, LoRA fine-tuning of Qwen models, and prediction aggregation across complementary scorers.

In practice

• Fine-tune Qwen models for German short answer scoring.
• Employ prediction aggregation for known question types.

Topics

• Short Answer Scoring
• German NLP
• Rubric-based Grading
• Qwen Models
• LoRA Fine-tuning
• Prediction Aggregation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Improving Answer Extraction in Context-based Question Answering Systems Using LLMs — Targeted fine-tuning of pre-trained LLMs significantly improves context-based question answering accuracy and relevance.
• Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays — Neuro-symbolic methods can enhance automated essay scoring with interpretability and fine-grained feedback.
• A BART-based approach with hierarchical strategy for Vietnamese abstractive multi-document summarization — A BART-based hierarchical strategy with golden summary-driven shortening improves Vietnamese multi-document abstractive summarization, achieving 0.2468 ROUGE2-F1.
• Use of Machine Learning techniques and Large Language Models for automatic evaluation of Celpe-Bras exam texts — BERT adaptations achieved the best automatic evaluation for Celpe-Bras texts, but at a higher computational cost.
• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity — LLMs, guided by a context-aware "Complexity Score" for prompt selection, significantly outperform fine-tuned models on inferentially complex, low-prevalence data extraction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.