Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies

2026-05-28 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A recent analysis on large language model (LLM) fine-tuning with Evolution Strategies (ES) investigates the phenomenon of prior-task forgetting. This research characterizes forgetting not as irreversible loss, but as performance drift that often recovers during ES training, and notes its occurrence also with reinforcement learning (RL) methods. The study attributes this drift to ES training dynamics, particularly random walk behavior in weakly constrained directions of the weight space. To counteract this, a new parameter-space regularization technique, Anchored Weight Decay (AWD), is introduced. AWD constrains optimization towards the initial model parameters, effectively stabilizing prior-task performance while preserving target-task performance. This method offers benefits similar to large ES population sizes but at a much lower computational cost, indicating that prior-task forgetting under ES is largely avoidable and positioning ES as a promising approach for continual learning in LLMs.

Key takeaway

For Machine Learning Engineers fine-tuning LLMs with Evolution Strategies, you should integrate Anchored Weight Decay (AWD) into your training pipeline. This technique effectively mitigates prior-task forgetting by stabilizing performance at a lower computational cost than increasing population sizes. Adopting AWD allows you to confidently apply ES for continual learning scenarios, ensuring new task performance without sacrificing prior knowledge. This makes ES a more robust and efficient option for your LLM development.

Key insights

Forgetting in LLM fine-tuning with Evolution Strategies is performance drift, not irreversible, and is largely avoidable using Anchored Weight Decay.

Principles

• Prior-task forgetting is performance drift, often recoverable.
• Weight space random walk causes performance drift.
• Parameter-space regularization stabilizes prior-task performance.

Method

Anchored Weight Decay (AWD) is a parameter-space regularization technique that constrains optimization toward the initial model parameters to stabilize prior-task performance during ES fine-tuning.

In practice

• Apply Anchored Weight Decay in ES fine-tuning.
• Explore ES for LLM continual learning.

Topics

• Evolution Strategies
• LLM Fine-tuning
• Catastrophic Forgetting
• Continual Learning
• Anchored Weight Decay
• Parameter Regularization

Best for: Research Scientist, NLP Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• Learning, Fast and Slow: Towards LLMs That Adapt Continually [R] — Combining slow parameter updates with fast context optimization enhances LLM adaptation and mitigates forgetting.
• Discrete Tilt Matching — DTM fine-tunes dLLMs by matching state-level unmasking posteriors under reward tilting, bypassing intractable sequence-level likelihoods.
• Bounding the Predictive Space: How Topological AI Solves Catastrophic Forgetting Through… — Topological AI uses prime-indexed embedding anchors to prevent catastrophic forgetting in LLMs while retaining cognitive flexibility.
• MIT's new fine-tuning method lets LLMs learn new skills without losing old ones — SDFT enables LLMs to learn new skills continually without forgetting old ones, using self-distillation and in-context learning.
• Continual Fine-Tuning with Provably Accurate and Parameter-Free Task Retrieval — Proteus offers provably accurate, parameter-free task retrieval for continual fine-tuning by leveraging distinct representation signatures.
• Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem — LLM unlearning can be reframed as an asymmetric two-task problem prioritizing retention over forgetting.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.