MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning

2026-07-08 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

MILES (Modular Instruction Memory with Learnable Selection) is a novel framework designed to enhance large language model (LLM) reasoning by accumulating reusable experience across sequential problems at test time. Unlike existing memory-based methods that store whole-solution templates or rely on heuristic step-level selection, MILES dynamically expands step-wise memory and applies correctness-optimized memory composition under realistic test-time constraints. The framework utilizes modular memory units, each comprising asymmetric pairs of sub-goal embeddings and sub-instructions, linked to a learnable selection head. This structure facilitates a coarse-to-fine retrieval mechanism: the coarse level manages memory expansion and gathers supervision from confident samples, while the fine stage employs learned selection heads to rerank candidates and guide reasoning for uncertain samples. MILES consistently matches or surpasses previous methods, demonstrating superior accuracy-efficiency tradeoffs, effectiveness, robustness, and transferability.

Key takeaway

For Machine Learning Engineers developing self-improving LLMs, consider integrating MILES's modular instruction memory. This framework allows your models to dynamically expand step-wise memory and apply correctness-optimized composition, moving beyond isolated problem-solving. You can achieve superior accuracy-efficiency tradeoffs and enhance reasoning robustness across diverse, sequential tasks, making your LLMs more adaptable and performant in real-world applications.

Key insights

MILES improves LLM reasoning by dynamically expanding modular memory and using learnable selection for correctness-optimized composition.

Principles

• Accumulate reusable experience across sequential problems.
• Optimize memory composition for final-answer correctness.
• Employ coarse-to-fine retrieval for memory management.

Method

MILES dynamically expands step-wise memory using modular units of sub-goal embeddings and sub-instructions. It applies a coarse-to-fine retrieval mechanism, training selection heads from confident samples and reranking candidates for uncertain ones.

In practice

• Enhance LLM reasoning in sequential problem-solving.
• Improve accuracy-efficiency tradeoffs for LLMs.
• Transfer memory-based reasoning across tasks.

Topics

• Large Language Models
• Reasoning
• Modular Memory
• Instruction Tuning
• Self-Improving AI
• Test-Time Adaptation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Open 32B w/ AutoMemory beats Opus: HOW? (Stanford) — Training LLM memory as an active cognitive skill via dual-loop optimization significantly boosts performance, even for older open-source models.
• ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models — Combining structured exploration, iterative refinement, explicit validation, and persistent memory significantly enhances LLM multi-step reasoning.
• MemTrace: Probing What Final Accuracy Misses in Long-Term Memory — MemTrace evaluates LLM long-term memory by probing knowledge points across controlled dimensions, revealing hidden failure modes beyond aggregated accuracy.
• Adaptive parallel reasoning: the next paradigm in efficient inference scaling — Adaptive Parallel Reasoning enables LLMs to dynamically manage parallel and serial operations for efficient, context-aware inference.
• Qwen3-8B w/ Self-Evolving Memory Harness on Financial Reasoning — A frozen LLM's performance can be significantly boosted by an external, structured memory and deterministic tool harness.
• MeMo's memory model lets teams upgrade their LLM without retraining it — and performance jumps 26% — MeMo uses a separate, smaller memory model to update LLM knowledge without retraining the main reasoning engine.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.