MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning

2026-07-09 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

MILES (Modular Instruction Memory with Learnable Selection for self-improving LLM reasoning) is a new framework designed to enhance large language model (LLM) reasoning at test time by accumulating reusable experience across sequential problems. Unlike existing memory-based methods that struggle with generalization or rely on unoptimized heuristic selection, MILES dynamically expands step-wise memory and applies correctness-optimized memory composition under realistic test-time constraints. It utilizes modular memory units, each comprising asymmetric pairs of sub-goal embeddings and sub-instructions, linked to a learnable selection head. This structure facilitates a coarse-to-fine retrieval mechanism, where the coarse stage expands memory and gathers supervision from confident samples, while the fine stage employs learned selection heads to rerank candidates and guide reasoning for uncertain samples. Experiments show MILES consistently matches or outperforms prior methods, demonstrating superior accuracy-efficiency tradeoffs, effectiveness, robustness, and transferability.

Key takeaway

For Machine Learning Engineers developing self-improving LLMs for sequential reasoning tasks, consider integrating MILES's modular instruction memory. This approach, with its learnable selection heads and coarse-to-fine retrieval, offers superior accuracy-efficiency tradeoffs compared to traditional memory methods. You should explore implementing dynamic memory expansion and correctness-optimized composition to enhance your models' generalization and overall reasoning capabilities at test time.

Key insights

MILES improves LLM reasoning by dynamically expanding modular instruction memory with learnable, correctness-optimized selection for sequential problem-solving.

Principles

• Accumulating reusable experience improves sequential problem-solving.
• Modular memory units with learnable selection enhance generalization.
• Correctness-optimized memory composition is crucial for LLM reasoning.

Method

MILES dynamically expands step-wise memory, maintaining modular units of sub-goal embeddings and sub-instructions with learnable selection heads. It uses a coarse-to-fine retrieval: coarse for memory expansion and supervision, fine for reranking and guiding reasoning.

In practice

• Apply modular memory for LLM test-time reasoning.
• Implement coarse-to-fine retrieval for memory selection.
• Optimize memory composition for final-answer correctness.

Topics

• LLM Reasoning
• Modular Memory
• Learnable Selection
• Instruction Memory
• Self-Improving LLMs
• Coarse-to-Fine Retrieval

Best for: AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning — MILES improves LLM reasoning by dynamically expanding modular memory and using learnable selection for correctness-optimized composition.
• Open 32B w/ AutoMemory beats Opus: HOW? (Stanford) — Training LLM memory as an active cognitive skill via dual-loop optimization significantly boosts performance, even for older open-source models.
• Efficient and Reasoning AI at the ACL 2026 — LLM efficiency and reasoning advancements address forgetting, speed, and memory, while benchmark reliability remains a critical concern.
• MeMo's memory model lets teams upgrade their LLM without retraining it — and performance jumps 26% — MeMo uses a separate, smaller memory model to update LLM knowledge without retraining the main reasoning engine.
• ReContext: A Smarter Way to Help LLMs Reason Over Long Contexts — ReContext improves LLM long-context reasoning by teaching memory organization, not just expanding context windows.
• Why force models to compute knowledge when they could just look it up? — Augmenting LLMs with conditional memory via lookup tables frees computational resources for enhanced reasoning.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.