Learning, Fast and Slow: Towards LLMs That Adapt Continually

2026-05-13 · Source: Machine Learning ML & Generative AI News · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Advanced, quick

Summary

A new "fast-slow learning" framework for large language models (LLMs) addresses the trade-off between parameter updates and in-context learning. This approach utilizes model parameters as "slow" weights and optimized context as "fast" weights. The fast weights absorb task-specific information from textual feedback, enabling rapid adaptation, while the slow weights maintain general reasoning behaviors and remain closer to the base model. This method, termed Fast-Slow Training (FST), demonstrates up to 3x greater sample efficiency than reinforcement learning (RL) alone on reasoning tasks, achieving higher performance asymptotes. FST also reduces catastrophic forgetting by keeping models up to 70% closer to the base LLM in terms of KL divergence, and preserves plasticity for better adaptation to subsequent tasks in continual learning scenarios.

Key takeaway

For AI Engineers developing adaptive LLMs, the Fast-Slow Training (FST) framework offers a robust solution to enhance sample efficiency and mitigate catastrophic forgetting. You should consider integrating FST's dual-weight approach to improve model plasticity and performance in dynamic, multi-task environments, especially where rapid adaptation and long-term knowledge retention are critical.

Key insights

Combining slow parameter updates with fast in-context learning improves LLM adaptation and reduces forgetting.

Principles

• Separate task-specific from general knowledge.
• Optimize context for rapid adaptation.
• Maintain base model integrity.

Method

Fast-Slow Training (FST) uses model parameters as "slow" weights and optimized context as "fast" weights, learning from textual feedback to adapt while preserving base model reasoning.

In practice

• Implement FST for sample-efficient LLM training.
• Apply FST in continual learning settings.
• Use FST to mitigate catastrophic forgetting.

Topics

• Fast-Slow Learning
• Large Language Models
• Catastrophic Forgetting
• Continual Learning
• In-context Learning

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Learning, Fast and Slow: Towards LLMs That Adapt Continually [R] — Combining slow parameter updates with fast context optimization enhances LLM adaptation and mitigates forgetting.
• Slow for AI Weights, Fast for AI Harness (FST) — FST jointly optimizes LLM parameters and dynamic context for more efficient and adaptable continual learning.
• MIT's new fine-tuning method lets LLMs learn new skills without losing old ones — SDFT enables LLMs to learn new skills continually without forgetting old ones, using self-distillation and in-context learning.
• Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments — Continual learning benchmarks must isolate online improvement from static model capabilities using expert-validated, latent-structure tasks.
• Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling — SMET stabilizes dynamic sparse training for LLMs by addressing cold-start issues and optimizing memory, making sparse pre-training practical.
• Training-Free Test-Time Contrastive Learning for Large Language Models — TF-TTCL improves LLM robustness under distribution shift by distilling self-generated inference experiences into explicit textual rules.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning ML & Generative AI News.