EssayCBM: Rubric-Aligned Concept Bottleneck Models for Transparent Essay Grading

2026-07-23 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, medium

Summary

EssayCBM is a rubric-aligned Concept Bottleneck Model (CBM) framework designed to enhance transparency and interpretability in automated essay grading. Unlike traditional black-box systems that directly predict grades, EssayCBM evaluates essays across eight explicit writing concepts, such as Thesis Clarity and Evidence Use, using dedicated prediction heads on an encoder. These concept scores, ranging from 0-4, form a transparent bottleneck, with a lightweight network then computing the final grade on a 0-5 scale solely from these concepts. This architecture allows instructors to adjust concept predictions and instantly see updated grades, facilitating human-in-the-loop evaluation. EssayCBM, available as an open-source web application, matches black-box model performance, with BERT achieving 81.14% accuracy and 62.38 F1, while providing actionable, concept-level feedback through an intuitive Streamlit interface. It supports four encoder backbones: BERT-base, RoBERTa-base, GPT-2, and BiLSTM.

Key takeaway

For educators and AI scientists developing automated assessment tools, EssayCBM offers a critical shift towards transparency. You can provide students with specific, rubric-aligned feedback, like identifying weak "Use of Evidence," rather than opaque scores. This enables targeted learning improvements and builds trust in AI grading. Consider deploying EssayCBM's open-source framework to integrate human-in-the-loop adjustments for high-stakes assessments, ensuring accountability and pedagogical value.

Key insights

EssayCBM provides transparent essay grading by predicting rubric-aligned concepts before assigning a final score.

Principles

• Grades should be traceable to explicit rubric dimensions.
• Interpretability can match black-box accuracy.
• Human-in-the-loop correction enhances trust.

Method

EssayCBM encodes essays, predicts 8 rubric-aligned concept scores (0-4) via independent classifiers, then aggregates these into a final grade (0-5) using a lightweight feed-forward network.

In practice

• Identify specific student weaknesses (e.g., Use of Evidence).
• Adjust teaching strategies based on class concept trends.
• Manually correct concept scores for grade appeals.

Topics

• Concept Bottleneck Models
• Automated Essay Grading
• Explainable AI
• Rubric-Aligned Feedback
• Human-in-the-Loop AI
• Educational Technology

Code references

• scott-f-zhang/CBM-Demo

Best for: NLP Engineer, AI Product Manager, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment — SHAP offers more faithful and transferable explanations for rubric-based scoring than LLM-generated rationales.
• AI is inflating student grades, and the effect points to outsourced work, not better learning — AI-driven grade inflation in writing and coding assignments indicates outsourced work, not improved student learning, eroding grade validity.
• Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays — Neuro-symbolic methods can enhance automated essay scoring with interpretability and fine-grained feedback.
• Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach — LLMs can reliably grade Linux/bash exams, especially with rubrics, but struggle with higher cognitive complexity questions.
• Using Learning Progressions to Guide AI Feedback for Science Learning — Learning progression-derived rubrics enable AI feedback quality comparable to expert-authored rubrics, enhancing scalability.
• PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback — PsyScore unifies essay scoring and adaptive feedback using a shared psychometric latent ability representation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.