Learning, Fast and Slow: Towards LLMs That Adapt Continually [R]

2026-05-13 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

A new framework, Fast-Slow Training (FST), addresses the trade-off between in-context learning and parameter-based updates in large language models (LLMs). Traditional parameter updates for downstream tasks often lead to catastrophic forgetting and reduced plasticity, while in-context learning, though fast, typically cannot match performance gains from parameter updates. FST introduces "slow" weights via model parameters and "fast" weights through optimized context, allowing LLMs to absorb task-specific information from textual feedback while preserving general reasoning behaviors in the base model. This approach is up to 3x more sample-efficient than reinforcement learning (RL) alone, achieves higher performance asymptotes, and reduces catastrophic forgetting by keeping models up to 70% closer to the base LLM in terms of KL divergence. FST also maintains plasticity, enabling more effective adaptation to subsequent tasks in continual learning scenarios.

Key takeaway

For research scientists developing continually adapting LLMs, FST offers a robust method to mitigate catastrophic forgetting and improve performance. By integrating fast in-context learning with slow parameter updates, you can achieve higher sample efficiency and maintain model plasticity across diverse tasks. Consider implementing FST to build more resilient and adaptable AI agents, especially in dynamic environments where task domains frequently change.

Key insights

FST combines slow parametric learning with fast in-context adaptation to enhance LLM performance and reduce forgetting.

Principles

• Combine learning at different timescales.
• Separate task-specific from general knowledge.

Method

FST uses model parameters as "slow" weights and optimized context as "fast" weights, learning from textual feedback to adapt to tasks while preserving base model integrity.

In practice

• Reduce catastrophic forgetting in LLMs.
• Improve sample efficiency in training.
• Enhance adaptation in continual learning.

Topics

• Large Language Models
• Fast-Slow Learning
• Catastrophic Forgetting
• Continual Learning
• In-Context Learning

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Learning, Fast and Slow: Towards LLMs That Adapt Continually — Combining slow parameter updates with fast in-context learning improves LLM adaptation and reduces forgetting.
• FOD#155: Continual Learning in LLMs: Why AI Models Need Sleep — LLMs require an offline consolidation phase, akin to "sleep," to integrate new experiences and prevent catastrophic forgetting.
• JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models — JumpLoRA uses adaptive sparsity via JumpReLU gating to reduce catastrophic forgetting in LLM continual learning.
• The Eigenvectors of AI: Shared LoRA Subspaces for Continual Learning — Task-specific LoRA adapters converge to a shared low-rank subspace, enabling massive storage and memory reduction.
• TFGN: Task-Free, Replay-Free Continual Pre-Training Without Catastrophic Forgetting at LLM Scale — TFGN enables replay-free, task-free continual pre-training for LLMs by architecturally separating read and write operations.
• Towards Continual Learning — Continual learning, crucial for human-like AI, involves updating models either through weight adjustments or external context/harness modifications.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.