Geometric Self-Distillation for Reasoning Generalization

2026-07-07 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, quick

Summary

GeoSD is a novel geometric self-distillation objective designed to counter predictive behavior drift in large language models during on-policy distillation. This post-training recipe addresses the issue where standard self-distillation, despite providing abundant teacher supervision, can degrade out-of-distribution (OOD) reasoning due to accumulated drift. GeoSD employs two complementary mechanisms: a Hellinger loss that scales teacher preferences by student overlap, and a proximal term that penalizes prediction drift from a recent checkpoint using Fisher-Rao distance. Both operate within the geometry of next-token distributions. Across mathematical reasoning benchmarks and three model families, GeoSD preserves in-distribution gains while improving average OOD accuracy by 5.7-8.6 points over base models, with consistent gains observed across model scales from 1.7B to 32B parameters. It achieves this by keeping alternative predictions in reach, unlike standard methods that confidently agree on wrong answers.

Key takeaway

For Machine Learning Engineers focused on improving large language model reasoning generalization, particularly out-of-distribution performance, consider implementing GeoSD. This geometric self-distillation approach can significantly boost OOD accuracy by 5.7-8.6 points while preserving in-distribution gains, addressing a key limitation of standard on-policy distillation. You should evaluate GeoSD as a post-training recipe to mitigate predictive drift and ensure your models maintain robust reasoning capabilities across diverse problem sets.

Key insights

GeoSD uses geometric self-distillation to prevent OOD reasoning drift in LLMs, improving accuracy by maintaining predictive alternatives.

Principles

• On-policy distillation can degrade OOD reasoning.
• Teacher confidence can lead to student drift.
• Geometric distances counter predictive behavior drift.

Method

GeoSD applies a Hellinger loss to scale teacher preferences and a proximal term using Fisher-Rao distance to penalize prediction drift, both in next-token distribution geometry.

In practice

• Improve OOD accuracy in LLMs.
• Preserve in-distribution gains during distillation.
• Avoid confident wrong answers in high-entropy states.

Topics

• Geometric Self-Distillation
• Large Language Models
• Out-of-Distribution Generalization
• Mathematical Reasoning
• On-Policy Distillation
• Fisher-Rao Distance

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• AI 101: "On-Policy Distillation Zeitgeist" — Self-distillation offers a scalable, on-policy method for LLMs to refine reasoning by self-critique and dense feedback.
• Geometric Foundation Model Distillation for Efficient Lunar 3D Reconstruction — Knowledge distillation can significantly compress large 3D foundation models for resource-constrained deployment while maintaining accuracy.
• Distribution Corrected Offline Data Distillation for Large Language Models — Offline distillation can be improved by correcting teacher-student distribution drift without costly online sampling.
• Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoning — Teacher predictive entropy is a crucial, often overlooked, signal for efficient LLM self-distillation.
• Local AI models destroyed by further Distillation — Self-distillation fails because teacher models impart memorized shortcuts, not reasoning, to student AI.
• SG-OPD: Sign-Gated On-Policy Distillation via Sign-Consistency Gating and Phased Teacher Sampling — SG-OPD improves on-policy distillation by verifying teacher reliability and selectively applying updates based on sign-consistency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.