Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards

2026-07-21 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

RLAES is a novel unified large language model (LLM) framework designed for automated essay scoring (AES) and feedback generation (AFG), leveraging reinforcement learning (RL) post-training. Unlike existing methods relying on prompt engineering or supervised fine-tuning, RLAES introduces Rubric-based Feedback Evaluation (RFE), an essay-grounded framework with 166 fine-grained binary rubric items and an LLM-as-judge, to systematically evaluate feedback quality. Building on RFE, the framework incorporates Adaptive Gated Feedback Optimization (AGFO), which activates rubric-based feedback rewards on demand during RL, reducing evaluation overhead. It also features Adjacent Contrastive Reasoning (ACR) to enhance ordinal score calibration by contrasting adjacent score levels. Experimental results on the ASAP benchmark show RLAES-AGFO achieved a QWK of 0.803, marking the best scoring performance among LLM-based methods, while maintaining feedback quality comparable to GPT-5.5. Code and datasets are publicly available.

Key takeaway

For Machine Learning Engineers developing automated essay scoring and feedback systems, RLAES offers a robust RL-based approach. You should consider integrating rubric-based feedback evaluation (RFE) and adaptive gated optimization (AGFO) to enhance both scoring accuracy and feedback quality. This method achieves strong performance (QWK = 0.803) while avoiding feedback degradation, providing a superior alternative to prompt engineering or supervised fine-tuning. Explore the public code and datasets to implement these techniques.

Key insights

The RLAES framework unifies LLM-based essay scoring and feedback generation via reinforcement learning with rubric-based rewards, outperforming prior methods.

Principles

• Feedback quality can be systematically evaluated using fine-grained rubrics.
• On-demand rubric rewards optimize RL training efficiency.
• Explicitly contrasting score levels improves ordinal calibration.

Method

RLAES jointly optimizes essay scoring and feedback generation using RL. It employs RFE for feedback evaluation, AGFO for adaptive rubric rewards, and ACR for ordinal score calibration.

In practice

• Implement RFE with 166 binary rubric items for feedback evaluation.
• Use AGFO to reduce RL evaluation overhead.
• Apply ACR for better ordinal score calibration in LLM-based scoring.

Topics

• Large Language Models
• Automated Essay Scoring
• Reinforcement Learning
• Feedback Generation
• Rubric Evaluation
• Ordinal Regression

Code references

• hellomuyi/RLAES

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards — RLAES unifies LLM-based essay scoring and feedback generation using reinforcement learning with rubric-based rewards for measurable quality.
• Automated Refinement of Essay Scoring Rubrics for Language Models via Reflect-and-Revise — Iterative self-reflection and revision significantly optimize LLM essay scoring rubrics for Automated Essay Scoring.
• Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays — Neuro-symbolic methods can enhance automated essay scoring with interpretability and fine-grained feedback.
• Investigating first-language bias in LLM-based automated essay scoring: A cross-prompt evaluation of an open-weight AI-model on TOEFL essays — A LoRA-adapted LLM shows robust cross-prompt generalization but significant first-language bias in automated essay scoring.
• Rubrics on Trial: Evolving Rubrics from a Single Query via Synthetic Pairwise Evidence — A query-only framework evolves validated LLM rubrics using synthetic response pairs, avoiding external data.
• Rubric-Based Rewards for RL — Rubric-based rewards extend RL to subjective LLM tasks by providing structured, interpretable evaluation criteria.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.