Understanding a Neural Network’s J-space

2026-07-09 · Source: Machine Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

Anthropic has introduced a new interpretability method that identifies a "J-space" within language models' internal representations. This J-space is a sparse, low-dimensional subspace derived from the Jacobian matrix, J = ∂L/∂h, which differentiates the logit vector L(h) with respect to hidden activations h. Each row of J represents a gradient direction ∇ₕLₜ, indicating how changes in activation affect a specific output token's logit. The method reveals that J-space, primarily active in middle network layers, occupies a tiny fraction of the full activation dimension ℝᵈ while retaining significant reportable information. It exists in pretrained models and evolves post-training, acquiring "Claude-like" perspectives. Causal interventions, such as swapping or injecting J-space components, reliably alter model behavior, demonstrating its functional importance. The authors compare J-space to a cognitive science "global workspace" due to its properties like verbal report, directed modulation, and flexible generalization, emphasizing its role as an inspectable and manipulable lens into model organization.

Key takeaway

For AI Scientists and NLP Engineers focused on mechanistic interpretability, understanding J-space provides a powerful new lens. You can inspect and causally manipulate a model's internal, reportable representations, offering direct insights into its reasoning and behavior. This enables more precise debugging, targeted safety interventions, and potentially greater control over model outputs by modifying specific conceptual components.

Key insights

J-space offers a causally important, low-dimensional, interpretable lens into language models' internal, reportable representations.

Principles

• Model interpretability can use gradient-derived subspaces.
• Sparse subspaces can preserve critical model information.
• Causal interventions validate functional importance.

Method

Compute Jacobian matrix J = ∂L/∂h from logits to hidden activations. Identify J-space as a sparse nonnegative mixture of gradient directions ∇ₕLₜ, then test via causal interventions.

In practice

• Inspect J-space coordinates to reveal active concepts.
• Manipulate J-space components to alter model behavior.
• Compare J-space across training stages for evolution.

Topics

• Neural Network Interpretability
• Language Model Representations
• Jacobian Matrix
• Causal Intervention
• Global Workspace Theory
• Model Controllability

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• A global workspace in language models — The J-space in Claude models functions as an emergent internal "global workspace" for higher-order reasoning and reportable thoughts.
• Resolving superposition in AI for interpretability and cross-modal alignment in patient-neuronal images — Sparse Autoencoders resolve neural network superposition, purifying latent space geometry for biological image interpretability.
• H-Probes: Extracting Hierarchical Structures From Latent Representations of Language Models — LLMs geometrically represent hierarchical structures like tree depth and pairwise distance in low-dimensional latent subspaces.
• Sparse probes and murky physics: a case study of interpretability challenges in a foundation model for continuum dynamics — Interpreting scientific foundation models reveals internal mechanisms are often inconsistent with known physics, posing evaluation challenges.
• Bridging Linguistic Structure and Mechanistic Interpretability for Conceptual Interpretation in Language Models — DSRA methodology links language model internal activations to definitional semantic roles, revealing how conceptual meaning is composed across layers.
• No Space Like J-Space — The J-space is a functional global workspace in LLMs, containing verbalizable representations crucial for internal reasoning and output determination.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning on Medium.