RETUYT-INCO at BEA 2026 Shared Task 2: Meta-prompting in Rubric-based Scoring for German

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

RETUYT-INCO participated in the BEA 2026 shared task "Rubric-based Short Answer Scoring for German," competing in tracks involving unseen answers and questions. To address the task's dynamic nature, the team developed "Meta-prompting," a method where a Large Language Model (LLM) generates a custom prompt from training examples, which is then used to grade new student answers. Alongside Meta-prompting, RETUYT-INCO also employed classic machine learning, fine-tuning open-source LLMs, and various other prompting techniques. Official results show the team placed 6th out of 8 in Track 1 with a QWK of 0.729, 4th out of 9 in Track 3 with a QWK of 0.674, and 4th out of 8 in Track 4 with a QWK of 0.49.

Key takeaway

For NLP Engineers developing automated grading systems for German short answers, consider integrating meta-prompting to handle evolving rubrics or unseen content. This approach allows your LLMs to dynamically generate task-specific prompts, potentially improving adaptability and performance compared to static prompting. Evaluate its effectiveness alongside fine-tuned open-source LLMs and classic machine learning methods for robust solutions.

Key insights

Meta-prompting enables LLMs to dynamically create prompts for rubric-based short answer scoring.

Principles

• LLMs can adapt to task variations via dynamic prompt generation.
• Combining diverse AI methods can improve task performance.

Method

An LLM generates a custom prompt based on training set examples. This dynamically created prompt is then applied to grade new student answers according to specific rubrics.

In practice

• Implement meta-prompting for adaptive grading systems.
• Explore hybrid approaches combining LLMs and classic ML.

Topics

• Meta-prompting
• Large Language Models
• Rubric-based Scoring
• German Language Processing
• Automated Grading
• Educational Applications

Best for: Research Scientist, AI Scientist, NLP Engineer, Prompt Engineer

Related on AIssential

• Feedback-to-Rubrics: Can We Learn Expert Criteria from Inline Comments? — Inline comments can be distilled into reusable natural-language rubrics, enhancing LLM-driven writing and review support.
• Meta AI prompts: Examples for common use cases | AI at Meta - AI at Meta — Meta AI, integrated into Meta apps, offers multimodal assistance and personalized experiences through intentional, clear prompting.
• Automatic Generation of Highlights for Academic Paper Via Prompt-based Learning — Prompt-based learning with LLMs can generate academic paper highlights effectively without extensive labeled data.
• Zuckerberg Is Cloning Himself? — Enhancing LLM performance requires strategic prompting, external knowledge integration, and agentic workflows, moving beyond base model limitations.
• Rubric-Based Rewards for RL — Rubric-based rewards extend RL to subjective LLM tasks by providing structured, interpretable evaluation criteria.
• Derivation Prompting: A Logic-Based Method for Improving Retrieval-Augmented Generation — Derivation Prompting uses logic-inspired rule application to enhance RAG generation, reducing LLM hallucinations and improving interpretability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.