Learning, Fast and Slow: Towards LLMs That Adapt Continually [R]

2026-05-15 · Source: Machine Learning ML & Generative AI News · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new "fast-slow learning framework" for large language models (LLMs) has been introduced, addressing the trade-off between parameter updates and in-context learning. Traditional parameter updates, often via reinforcement learning (RL), can lead to catastrophic forgetting and reduced plasticity, while in-context learning, like prompt optimization, offers rapid adaptation but typically lower performance gains. This framework proposes using model parameters as "slow" weights and optimized context as "fast" weights. The Fast-Slow Training (FST) method is up to 3x more sample-efficient than slow-only learning (RL) on reasoning tasks, achieving higher performance. FST also reduces catastrophic forgetting by keeping models up to 70% closer to the base LLM via lower KL divergence, preserving plasticity for subsequent tasks and excelling in continual learning scenarios where parameter-only RL stalls.

Key takeaway

For research scientists developing adaptive LLMs, consider integrating the fast-slow learning framework to overcome limitations of traditional parameter-only updates. This approach can significantly improve sample efficiency and reduce catastrophic forgetting, allowing your models to adapt more effectively to new tasks and perform robustly in continual learning environments. Prioritize methods that maintain base model integrity while enabling rapid, context-driven adaptation.

Key insights

Combining slow parameter updates with fast context optimization enhances LLM adaptation and mitigates forgetting.

Principles

• Separate task-specific from general reasoning.
• Fast weights absorb specific task information.
• Slow weights maintain general reasoning.

Method

Fast-Slow Training (FST) uses model parameters as "slow" weights and optimized context as "fast" weights, learning from textual feedback to adapt LLMs.

In practice

• Achieve 3x higher sample efficiency.
• Reduce catastrophic forgetting by 70%.
• Improve adaptation to new tasks.

Topics

• Large Language Models
• Catastrophic Forgetting
• Continual Learning
• Fast-Slow Learning Framework
• Fast-Slow Training

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Learning, Fast and Slow: Towards LLMs That Adapt Continually — Combining slow parameter updates with fast in-context learning improves LLM adaptation and reduces forgetting.
• Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies — Forgetting in LLM fine-tuning with Evolution Strategies is performance drift, not irreversible, and is largely avoidable using Anchored Weight Decay.
• MIT's new fine-tuning method lets LLMs learn new skills without losing old ones — SDFT enables LLMs to learn new skills continually without forgetting old ones, using self-distillation and in-context learning.
• Towards Automated Knowledge Transfer in Evolutionary Multitasking via Large Language Models — LLMs can autonomously design efficient and effective knowledge transfer models for multi-task optimization.
• Learning Perturbations to Extrapolate Your LLM — Learnable continuous perturbations in embedding space significantly improve LLM out-of-domain extrapolation.
• Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem — LLM unlearning can be reframed as an asymmetric two-task problem prioritizing retention over forgetting.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning ML & Generative AI News.