InfoSFT: Learn More and Forget Less with Information-Aware Token Weighting

2026-05-14 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

InfoSFT is a novel weighting scheme designed to improve supervised fine-tuning (SFT) of large language models (LLMs) by focusing learning signals on maximally informative tokens. Standard SFT often overfits to low-likelihood samples, leading to policy shifts and degradation of prior capabilities. While existing methods filter or down-weight such data, they risk suppressing novel behaviors. InfoSFT addresses this by concentrating training updates on medium-confidence tokens, which are neither overly familiar nor too unlikely to cause instability. This method requires only a one-line modification to the standard token-wise loss and has shown improved generalization over vanilla SFT and likelihood-weighted baselines across math, code, and chain-of-thought tasks, while also better preserving pre-existing model capabilities.

Key takeaway

For AI Engineers and Research Scientists developing or fine-tuning LLMs, integrating InfoSFT into your SFT pipeline can significantly improve model generalization and stability. By focusing learning on optimally informative tokens, you can achieve better performance across tasks like math, code, and chain-of-thought, while simultaneously preserving the model's pre-existing capabilities more effectively than with standard SFT or likelihood-weighted approaches. Consider this one-line modification to enhance your LLM training outcomes.

Key insights

InfoSFT improves LLM fine-tuning by weighting tokens based on informativeness, balancing novelty and stability.

Principles

• Uniform SFT fitting can lead to overfitting.
• Focus learning on medium-confidence tokens.
• Preserve prior capabilities during fine-tuning.

Method

InfoSFT modifies the standard token-wise SFT loss with a principled weighting scheme that prioritizes maximally informative, medium-confidence tokens to enhance generalization and stability.

In practice

• Apply InfoSFT for improved LLM generalization.
• Use InfoSFT to mitigate SFT policy shifts.
• Enhance fine-tuning across diverse tasks.

Topics

• Supervised Fine-Tuning
• Token Weighting
• InfoSFT
• Generalization Improvement
• Pre-existing Capability Preservation

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• No more Catastrophic Forgetting in SFT — EKSFT selectively masks high-entropy or high-KL divergence tokens during SFT to mitigate catastrophic forgetting and preserve generalization.
• Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability — Reasoning SFT can generalize cross-domain, but it is conditional on optimization, data quality, and model capability.
• Nous Research Releases Token Superposition Training to Speed Up LLM Pre-Training by Up to 2.5x Across 270M to 10B Parameter Models — Token Superposition Training accelerates LLM pre-training by modifying the training loop without changing model architecture or inference.
• Learning Perturbations to Extrapolate Your LLM — Learnable continuous perturbations in embedding space significantly improve LLM out-of-domain extrapolation.
• MIT's new fine-tuning method lets LLMs learn new skills without losing old ones — SDFT enables LLMs to learn new skills continually without forgetting old ones, using self-distillation and in-context learning.
• Learning What to Learn: Stage-Specific Data Sets for SFT-then-RL in Small Language Model Reasoning — Data strategy for SFT-then-RL pipelines should align with each stage's distinct role in skill acquisition and consolidation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.