Structure Before Collapse: Transient semantic geometry in next-token prediction

2026-06-25 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

The paper "Structure Before Collapse: Transient semantic geometry in next-token prediction" by Yize Zhao, Isabel Papadimitriou, and Christos Thrampoulidis investigates a paradox in language models. Despite being trained predominantly with one-hot labels, which Neural Collapse theory predicts should lead to symmetric, semantically undifferentiated representations, these models clearly learn latent structural features. Through three synthetic controlled settings, the authors demonstrate that semantic geometry emerges early in training, causing representations to cluster by shared attributes even without explicit supervision. This emergent structure is transient; with sufficient capacity and training time, the model eventually converges to the predicted symmetric state. The study employs Gram matrix analysis to examine this phase transition and proposes a preliminary modification to the commonly used unconstrained features model to better capture the emergent semantic geometry.

Key takeaway

For AI Scientists optimizing language model training, recognize that valuable semantic structure emerges early but is transient. Your models may learn rich semantic geometry initially, only for it to collapse into symmetric, less semantically useful representations with prolonged training. Consider strategies to capture or stabilize this emergent geometry, perhaps by adjusting training duration, implementing regularization, or exploring architectural modifications to preserve these critical latent features.

Key insights

Semantic structure in next-token prediction LMs emerges transiently despite one-hot training, before collapsing to symmetric representations.

Principles

• Neural Collapse predicts symmetric representations in one-hot classification.
• Semantic geometry can emerge without explicit supervision.
• Early training phases can exhibit transient, structured representations.

Method

Investigated semantic structure emergence using synthetic settings and Gram matrix analysis. Proposed a preliminary modification to the unconstrained features model.

In practice

• Analyze representation dynamics with Gram matrix.
• Consider early training phases for semantic structure.
• Explore model modifications to preserve emergent geometry.

Topics

• Next-token prediction
• Neural Collapse
• Semantic Geometry
• Language Models
• Representation Learning
• Gradient Descent Dynamics

Code references

• gregorbachmann/Next-Token-Failures
• wyzjack/CNTP

Best for: Research Scientist, AI Scientist

Related on AIssential

• Bounding the Predictive Space: How Topological AI Solves Catastrophic Forgetting Through… — Topological AI uses prime-indexed embedding anchors to prevent catastrophic forgetting in LLMs while retaining cognitive flexibility.
• Uncovering Symmetry Transfer in Large Language Models via Layer-Peeled Optimization — LLM training implicitly transfers data symmetries into structured geometric configurations within model parameters.
• Attention is Just Another Name for Coupling?: A Fast-Slow ODE Perspective on Hierarchical Pretraining — Hierarchical pretraining can integrate fast token-rate attention with slower, downsampled sequence coupling via a fast-slow ODE framework.
• Predict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning — Combining JEPA latent prediction with MLM improves text encoder embedding uniformity and semantic-lexical balance without immediate linear-probe accuracy gains.
• The New Geometry of Intelligence #ai — Neural networks under capacity pressure form a "crystalline latent structure" using "tight frames" for efficient feature representation.
• Tracking Representation Dynamics in Large Language Models with Persistent Homology — Persistent homology uncovers internal representation dynamics in LLMs during fine-tuning, showing early topological reorganization and objective-specific trajectories.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.