Structure Before Collapse: Transient semantic geometry in next-token prediction

2026-06-25 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, quick

Summary

The paper "Structure Before Collapse: Transient semantic geometry in next-token prediction" investigates a paradox in language models (LMs) where they learn latent semantic structure despite being trained predominantly with one-hot labels. This training regime, according to Neural Collapse theory, should push representations to be symmetrically separated, ignoring semantic similarities. Using three synthetic controlled settings, researchers found that semantic geometry, where representations cluster by shared attributes, emerges early in training without explicit supervision. However, this structure is transient; with sufficient model capacity and training time, the representations eventually collapse to the predicted symmetric state. The study employs Gram matrix analysis to examine this phase transition and proposes a preliminary modification to the unconstrained features model to better capture the emergent semantic geometry.

Key takeaway

For AI Scientists and NLP Engineers designing or fine-tuning language models, understanding the transient nature of semantic geometry is crucial. Your model's capacity and training duration directly influence whether it retains rich, clustered semantic representations or collapses to a symmetric, less semantically nuanced state. Consider exploring architectural modifications, like the proposed unconstrained features model adjustment, or specific training strategies to preserve valuable emergent semantic structures for improved model performance and interpretability.

Key insights

Language models transiently learn semantic structure from one-hot labels before representations collapse to a symmetric, non-semantic state.

Principles

• Neural Collapse predicts symmetric representations from one-hot labels.
• Semantic geometry emerges without explicit supervision.
• Latent semantic structure is transient in LMs.

Method

Investigated semantic structure emergence and collapse using three synthetic controlled settings with latent semantic factors and distinct one-hot labels, analyzed via Gram matrix.

Topics

• Language Models
• Next-Token Prediction
• Neural Collapse
• Semantic Geometry
• Gram Matrix Analysis
• Latent Structure

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Structure Before Collapse: Transient semantic geometry in next-token prediction — Semantic structure in next-token prediction LMs emerges transiently despite one-hot training, before collapsing to symmetric representations.
• The New Geometry of Intelligence #ai — Neural networks under capacity pressure form a "crystalline latent structure" using "tight frames" for efficient feature representation.
• Tracking Representation Dynamics in Large Language Models with Persistent Homology — Persistent homology uncovers internal representation dynamics in LLMs during fine-tuning, showing early topological reorganization and objective-specific trajectories.
• AI 101: What’s So Magical About Embeddings? — Embeddings transform discrete tokens into dense vectors, encoding meaning through geometric distance in a continuous space.
• Attention is Just Another Name for Coupling?: A Fast-Slow ODE Perspective on Hierarchical Pretraining — Hierarchical pretraining can integrate fast token-rate attention with slower, downsampled sequence coupling via a fast-slow ODE framework.
• Bounding the Predictive Space: How Topological AI Solves Catastrophic Forgetting Through… — Topological AI uses prime-indexed embedding anchors to prevent catastrophic forgetting in LLMs while retaining cognitive flexibility.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.