Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution

2026-04-16 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

The ReMe (Remember Me, Refine Me) framework introduces a dynamic procedural memory system for large language model (LLM) agents, addressing the limitations of static, append-only memory archives. Developed by researchers from Shanghai Jiao Tong University and Alibaba Group's Tongyi Lab, ReMe enables experience-driven agent evolution through three core mechanisms: multi-faceted distillation for extracting fine-grained experiences, context-adaptive reuse via scenario-aware indexing, and utility-based refinement for autonomously managing a high-quality experience pool. Experiments on BFCL-V3 and AppWorld benchmarks demonstrate ReMe's state-of-the-art performance, with Qwen3-8B equipped with ReMe outperforming a larger, memoryless Qwen3-14B model, achieving average gains of 8.83% in Avg@4 and 7.29% in Pass@4. This suggests that self-evolving memory offers a computation-efficient pathway for lifelong learning in LLM agents. The code and reme.library dataset are publicly released.

Key takeaway

For NLP Engineers and Research Scientists developing LLM agents, ReMe demonstrates that investing in dynamic memory management can yield significant performance improvements and computational efficiencies. You should consider integrating multi-faceted distillation and utility-based refinement into your agent architectures to enable self-evolving learning, potentially allowing smaller models to achieve or exceed the performance of larger, memory-intensive baselines. This approach offers a pathway to more robust and resource-efficient agent development.

Key insights

ReMe dynamically manages LLM agent procedural memory, enabling smaller models to surpass larger, memoryless counterparts through experience-driven evolution.

Principles

• Procedural memory requires dynamic management, not passive accumulation.
• Memory quality can substitute for model scale in LLM agents.
• Fine-grained experience extraction enhances knowledge transfer.

Method

ReMe employs multi-faceted distillation for experience extraction, context-adaptive reuse via scenario-aware indexing, and utility-based refinement for dynamic memory pool optimization.

In practice

• Implement multi-faceted distillation for success patterns, failure triggers, and comparative insights.
• Use usage scenario indexing for context-adaptive experience retrieval.
• Apply utility-based deletion to prune low-utility or outdated memories.

Topics

• Procedural Memory
• LLM Agents
• Experience Distillation
• Context-Adaptive Reuse
• Utility-Based Refinement

Code references

• agentscope-ai/ReMe

Best for: NLP Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• AEL: Agent Evolving Learning for Open-Ended Environments — AEL enables LLM agents to learn from experience by dynamically selecting memory retrieval policies and integrating reflective causal insights.
• EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments — LLM agents need version-aware memory to reliably adapt to dynamic, evolving real-world environments.
• AI Agents of the Week: Papers You Should Know About — Decoupling LLM reasoning from learning via external, hierarchical procedural memory enables efficient, continual agent adaptation.
• Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents — Experience compression unifies LLM agent memory, skills, and rules, crucial for scalable, long-horizon deployments.
• Memp: Exploring Agent Procedural Memory — Procedural memory, dynamically built and updated, significantly enhances LLM agent performance and efficiency across diverse tasks.
• MemRefine: LLM-Guided Compression for Long-Term Agent Memory — LLM-guided memory compression, prioritizing factual content over surface similarity, effectively manages agent memory within fixed budgets.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.