From Texts to Scores: Tracing the Emergence of Essay Quality Representations in Large Language Models

2026-06-19 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

A study systematically analyzed the hidden representations of eight Large Language Models (LLMs) across two English essay datasets (ASAP++, CSEE) and one Portuguese dataset (ENEM) to understand their internal mechanisms for Automated Essay Scoring (AES). Researchers found consistent evidence that essay quality information is encoded in a linearly accessible form within LLM representations. These representations emerge progressively across layers, remain robust across prompting strategies, and partially transfer across essay prompts despite differing scoring rubrics. The analysis, using linear probing, cross-prompt generalization, dimensionality reduction, and neuron-level analyses, also revealed that nonlinear probes offer only marginal improvements, indicating high linear decodability. Furthermore, specific "essay scoring neurons" were identified, whose activations strongly correlate with essay scores and whose layer-wise distribution shifts with essay length, with deeper layers being more critical for longer essays.

Key takeaway

For Machine Learning Engineers developing Automated Essay Scoring (AES) systems, understanding that LLMs encode essay quality linearly and progressively across layers is crucial. You should focus on linear probing techniques for efficient and interpretable scoring, as nonlinear methods offer minimal gains. Furthermore, consider analyzing "essay scoring neurons" and adapting your model's layer utilization based on essay length to optimize performance and gain deeper insights into how your LLM assesses writing quality.

Key insights

LLMs encode essay quality information in linearly accessible, progressively emerging representations, with specific "scoring neurons" adapting to essay length.

Principles

• Essay quality is linearly decodable from LLM representations.
• Quality representations emerge progressively across layers.
• Specific "essay scoring neurons" exist.

Method

The study used linear probing, cross-prompt generalization, dimensionality reduction, and neuron-level analyses to trace essay quality representations in LLMs across English and Portuguese datasets.

In practice

• Identify "essay scoring neurons" for AES interpretability.
• Utilize linear probes for efficient essay scoring.
• Consider deeper layers for longer essay analysis.

Topics

• Large Language Models
• Automated Essay Scoring
• LLM Interpretability
• Hidden Representations
• Linear Probing
• Essay Quality

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• From Texts to Scores: Tracing the Emergence of Essay Quality Representations in Large Language Models — LLMs encode linearly accessible essay quality representations, identifiable through specific neurons and layer-wise shifts.
• Rhetorical Questions in LLM Representations: A Linear Probing Study — LLMs represent rhetorical questions heterogeneously, with multiple linear directions capturing distinct rhetorical cues.
• Creating and Evaluating K-12 GenAI Assessment Graders Through Context Engineering — Context engineering enables LLMs to reliably grade K-12 assessments, particularly for objective subjects, but human oversight is vital.
• MADRAG: Multi-Agent Debate with Retrieval-Augmented Generation for Training-Free Analytic Essay Scoring — Multi-agent debate with retrieval augmentation offers a training-free, reliable approach to analytic essay scoring, outperforming prompt-based LLMs.
• LLM Predictive Scoring and Validation: Inferring Experience Ratings from Unstructured Text — LLMs can directionally predict experience ratings from unstructured text, revealing distinct evaluative constructs.
• Label-Aware Pseudo-Training Sample Generation for Text Classification — A new data augmentation method uses LLMs to subtly alter sentences, improving NLP performance with limited data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.