How are linear representations learned? Exact solutions to the dynamics of abstraction

2026-07-09 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new framework provides exact solutions for the dynamics of "abstraction"—the emergence of linear concept representations—during neural network training. In minimal linear networks, the study reveals that abstraction at the end of learning is jointly determined by data and target geometry, improves with network depth, and its maximum level is controlled by initialization scale. Extending to nonlinear networks, the research analyzes how activation functions like erf and ReLU affect these dynamics. It proves an "attenuation law," demonstrating that both nonlinearities weaken abstraction in activations compared to preactivations. This law is supported by observations in open models such as DINOv3 and Gemma 4, and the theory is applied to enhance linear probe generalization in Large Language Models.

Key takeaway

For Machine Learning Engineers designing or interpreting deep neural networks, understanding abstraction dynamics is crucial. You should consider network depth and initialization scale to optimize linear representation quality. Be aware that nonlinearities attenuate abstraction in activations, impacting linear probe effectiveness. Apply this theory to improve linear probe generalization in your LLM projects and refine interpretability efforts.

Key insights

This work provides exact solutions detailing how linear concept representations, or abstraction, dynamically emerge and evolve during neural network training.

Principles

• Abstraction depends on data/target geometry.
• Network depth enhances abstraction.
• Initialization scale limits abstraction peak.

Method

A framework derives exact solutions for abstraction dynamics in minimal linear networks, then extends analysis to nonlinear networks to study activation function effects.

In practice

• Improve LLM linear probe generalization.
• Guide neural network interpretability.
• Inform network control methods.

Topics

• Linear Representations
• Abstraction Dynamics
• Neural Network Training
• Interpretability
• Linear Probes
• Large Language Models

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Learning Dynamics Reveal a Hierarchy of Weight-Induced Layerwise Gram Metrics — Gradient descent in ReLU networks can be modeled as collective dynamics in training-set space, mediated by layer-wise Gram operators.
• How Causal Abstraction Underpins Computational Explanation — Causal abstraction offers a robust framework for defining computational implementation and explaining system behavior, especially in AI.
• Why Do Neural Networks Actually Work? Four Questions Nobody Has Fully Answered — Understanding neural networks requires addressing four distinct, partially answered questions rather than a single unified explanation.
• When Probing Accuracy Saturates, Fragility Resolves: A Complementary Metric for LLM Pre-Training Analysis — Fragility, a new metric, reveals LLM pre-training dynamics invisible to standard probing accuracy.
• 2 Years of My Research Explained in 13 Minutes — Discrete representations enhance learning efficiency and adaptation in reinforcement learning, especially under capacity constraints or dynamic environments.
• Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation — Statistical physics reveals deep learning's feature learning limits and optimal specialization in the interpolation regime.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.