A Quantitative Characterization of Forgetting in Post-Training

2026-03-12 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Krishnakumar Balasubramanian and Shiva Prasad Kasiviswanathan's March 2026 paper, "A Quantitative Characterization of Forgetting in Post-Training," investigates the mechanisms behind forgetting in continually post-trained generative models. Building on a two-mode mixture abstraction from Chen et al. (2025), the authors formalize forgetting into two types: mass forgetting, where the old task's mixture weight collapses, and old-component drift, where an existing correct component shifts. They prove that forward-KL objectives lead to mass forgetting, while reverse-KL objectives avoid it and cause old-component drift that decays exponentially with mode separation. The study also quantifies how replay interacts with these objectives, showing it modifies the training distribution for forward-KL but prevents old-mode starvation for reverse-KL. Finally, the paper analyzes SDFT, TTT-Discover, and OAPL, deriving conditions for retaining old mass and controlling drift.

Key takeaway

For AI Researchers developing continual learning strategies, understanding the interplay between divergence objectives and replay is critical. Your choice of forward-KL versus reverse-KL directly impacts whether old knowledge is completely forgotten or merely drifts. Prioritize reverse-KL and strategic replay to preserve past learning effectively, especially when fine-tuning generative models on new data.

Key insights

Forgetting in generative models is quantifiable, driven by divergence direction, geometric overlap, and sampling.

Principles

• Forward-KL objectives drive old task weights to zero.
• Reverse-KL objectives avoid mass forgetting.
• Drift decays exponentially with mode separation.

Method

The paper formalizes forgetting using a two-mode mixture abstraction, distinguishing between mass forgetting and old-component drift, and analyzes divergence objectives and replay mechanisms.

In practice

• Use reverse-KL to mitigate mass forgetting.
• Employ replay to prevent old-mode starvation.
• Consider mode separation for drift control.

Topics

• Continual Learning
• Generative Models
• Catastrophic Forgetting
• KL Divergence
• Replay Mechanisms

Best for: AI Researcher, AI Scientist, Research Scientist

Related on AIssential

• Forgetting is Not Erasure: Recovering Latent Knowledge via Transport Keys — Catastrophic forgetting often reflects interface drift, not erasure, allowing latent knowledge recovery via alignment.
• Continual Learning in Modern Hopfield Networks with an Application to Diffusion Models — High-energy, outlier-like samples are more prone to forgetting in continual learning for generative models, and replay is most effective for them.
• Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks — Sleep-inspired replay after sequential training prevents catastrophic forgetting in neural networks, enabling continuous learning.
• From Order to Distribution: A Spectral Characterization of Forgetting in Continual Learning — Task distribution geometry, not just task order, fundamentally governs forgetting rates in continual learning.
• Retrievable Gradients: Continual Post-Training Without Cumulative Weight Drift — ReGrad enables drift-free continual post-training by treating gradients as retrievable, temporary knowledge units.
• Sparsity, Superposition, and Forgetting: A Mechanistic Study of Representation Retention in Continual Learning — The study reveals that superposition's impact on forgetting in CL is nuanced, interacting with representation strength and capacity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.