From Texts to Scores: Tracing the Emergence of Essay Quality Representations in Large Language Models

2026-06-18 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

A systematic analysis investigated how eight Large Language Models (LLMs) encode essay quality for Automated Essay Scoring (AES) across two English datasets (ASAP++, CSEE) and one Portuguese dataset (ENEM). Researchers employed linear probing, cross-prompt generalization, dimensionality reduction, and neuron-level analyses. Findings consistently show that essay quality information is encoded in a linearly accessible form within LLM representations, emerging progressively across layers and remaining robust to prompting strategies. This information partially transfers across essay prompts despite rubric differences. Nonlinear probes offered only marginal improvements. The study also identified specific "essay scoring neurons" whose activations correlate with scores and whose layer-wise distribution shifts with essay length, providing insights into LLM-based AES interpretability.

Key takeaway

For NLP Engineers and AI Scientists developing Automated Essay Scoring systems, understanding the internal mechanisms of LLMs is crucial. This research indicates that essay quality is linearly encoded within LLM representations, suggesting that focusing on these hidden states and "essay scoring neurons" can lead to more interpretable and robust AES models. You should investigate layer-wise representation shifts and neuron activations to refine model design and prompt engineering for improved scoring accuracy and transparency.

Key insights

LLMs encode linearly accessible essay quality representations, identifiable through specific neurons and layer-wise shifts.

Principles

• Essay quality data is linearly decodable in LLM representations.
• Quality representations emerge progressively across layers.
• "Essay scoring neurons" correlate with scores and shift by essay length.

Method

Systematic analysis using linear probing, cross-prompt generalization, dimensionality reduction, and neuron-level analyses on LLM hidden representations.

In practice

• Analyze LLM hidden states for task-specific feature encoding.
• Identify "scoring neurons" to enhance AES interpretability.
• Evaluate representation robustness across diverse prompts.

Topics

• Automated Essay Scoring
• Large Language Models
• Model Interpretability
• Neural Representations
• Linear Probing
• Natural Language Processing

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Automatic Evaluation of Enem Essays: A Comparative Analysis between Feature Engineering and Transformers — Regression formulations are generally more suitable for ordinal essay scoring than multiclass classification.
• Rhetorical Questions in LLM Representations: A Linear Probing Study — LLMs encode rhetorical questions via multiple linear directions, emphasizing distinct cues rather than a single shared representation.
• Text-as-Signal: Quantitative Semantic Scoring with Embeddings, Logprobs, and Noise Reduction — Text can be transformed into operational, continuous semantic data points for AI engineering via embeddings and logprob scoring.
• Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews — AI review utility stems from textual justification, not just scalar scores, requiring text-centric evaluation.
• Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays — Neuro-symbolic methods can enhance automated essay scoring with interpretability and fine-grained feedback.
• One Language, Two Scripts: Probing Script-Invariance in LLM Concept Representations — SAE features capture abstract semantic meaning independent of surface-level orthography and tokenization.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.