Mechanistic Interpretability of Cognitive Complexity in LLMs via Linear Probing using Bloom's Taxonomy

2026-07-20 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

This study investigates the internal neural representations of cognitive complexity in Large Language Models (LLMs) using Bloom's Taxonomy as a hierarchical framework. By analyzing high-dimensional activation vectors from the residual streams of four state-of-the-art open-weight LLMs, the research demonstrates that linear classifiers achieve approximately 95% mean accuracy across all six Bloom levels (Remember, Understand, Apply, Analyze, Evaluate, Create). This indicates that cognitive level is encoded in a linearly accessible subspace of the model's representations. The findings show that this "Cognitive Separability Onset" (CSO) emerges early in the forward pass, specifically around layer 5 for most models, and remains stable in deeper layers. A curated dataset of 1,128 questions, ranging from 3 to 39 words, was used. Control experiments confirmed that this separability is not attributable to surface-level lexical patterns, as TF-IDF and sentence embedding baselines achieved significantly lower accuracies of 73% and 61% respectively. Misclassifications primarily occur between adjacent Bloom levels, reflecting the taxonomy's ordinal structure.

Key takeaway

For AI Scientists and Machine Learning Engineers evaluating LLM capabilities, recognize that models internally encode cognitive complexity aligning with Bloom's Taxonomy. You should consider probing internal representations, especially in early-to-mid layers (e.g., around layer 5), to understand how models process different cognitive demands. This approach helps distinguish between superficial performance and genuine reasoning, informing the development of more transparent and robust AI systems. Future work could involve causal intervention studies.

Key insights

LLMs encode cognitive complexity in a linearly separable latent space, emerging early in processing.

Principles

• Cognitive depth is a structural feature of LLM representations.
• Internal representations reflect Bloom's Taxonomy's ordinal structure.
• Cognitive separability emerges early in the forward pass.

Method

Extract activation vectors from LLM residual streams, then train layer-wise linear probes (Logistic Regression) to classify Bloom levels, evaluating accuracy and confusion matrices.

In practice

• Use Bloom's Taxonomy to audit LLM internal mechanics.
• Probe early layers for cognitive complexity signals.
• Distinguish rote memorization from genuine reasoning.

Topics

• Mechanistic Interpretability
• Large Language Models
• Bloom's Taxonomy
• Linear Probing
• Cognitive Complexity
• Neural Representations

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Student

Related on AIssential

• Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study — Self-harm representations in LLMs crystallize late in networks but are geometrically unstable and distributed across activation space.
• Quick Paper Review: "There Will Be a Scientific Theory of Deep Learning" — Learning mechanics, a theory of deep learning training dynamics, is proposed as an emerging scientific framework.
• From Isolated Tasks to Structured Capabilities: A Multilayer Taxonomy for Large Language Models — A multi-layer taxonomy organizes LLM capabilities, moving beyond task-centric evaluation to reveal underlying cognitive structures and research gaps.
• Mechanistic Interpretability: Peeking Inside an LLM — Mechanistic interpretability dissects LLM internal states to understand and control their emergent behaviors and knowledge.
• Mechanistic Decoding of Cognitive Constructs in LLMs — LLMs encode complex emotions like jealousy as structured linear combinations of psychological antecedents.
• Nobody Knows What’s Inside a Neural Network. A Few People Are Trying to Find Out. — A small group of researchers is undertaking "biology on language models," dissecting them like organisms to uncover their internal mechanisms, a process revealing unexpected findings.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.