Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents

2026-07-25 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

AgeMem is a unified framework for large language model (LLM) agents that integrates long-term memory (LTM) and short-term memory (STM) management directly into the agent's policy. It exposes memory operations as tool-based actions, allowing LLM agents to autonomously decide when and what information to store, retrieve, update, summarize, or discard. To train these behaviors, AgeMem employs a three-stage progressive reinforcement learning strategy and a step-wise Group Relative Policy Optimization (GRPO) to handle sparse rewards. Experiments on five long-horizon benchmarks, including ALFWorld, SciWorld, PDDL, BabyAI, and HotpotQA, demonstrate AgeMem's consistent outperformance over strong memory-augmented baselines, achieving average performance gains of 49.59% on Qwen2.5-7B-Instruct and 23.52% on Qwen3-4B-Instruct over no-memory baselines. It also shows higher memory quality (MQ scores of 0.533 and 0.605) and more efficient context usage, reducing prompt tokens by 3.1% to 5.1% compared to RAG variants.

Key takeaway

For AI Engineers developing LLM agents for long-horizon tasks, consider integrating AgeMem's unified memory management framework to overcome context window limitations and improve reasoning. By exposing memory operations as explicit tools and training with a three-stage progressive reinforcement learning strategy, you can enable agents to autonomously manage LTM and STM, leading to enhanced task performance, higher-quality memory, and more efficient token usage. Evaluate its applicability for complex, multi-step reasoning workflows to build more adaptive and scalable LLM agents.

Key insights

LLM agents can learn unified long-term and short-term memory management via tool-based actions and progressive reinforcement learning.

Principles

• Memory operations as explicit tools.
• Progressive RL for complex behaviors.
• Unified LTM/STM management is crucial.

Method

AgeMem uses a three-stage progressive RL strategy: LTM construction, STM control under distractors, and integrated reasoning. Step-wise GRPO addresses sparse rewards by broadcasting terminal advantages to all trajectory steps.

In practice

• Implement memory operations as LLM tools.
• Use multi-stage RL for memory training.
• Apply GRPO for sparse reward tasks.

Topics

• LLM Agents
• Memory Management
• Reinforcement Learning
• Long-Term Memory
• Short-Term Memory
• Tool-based AI

Code references

• WujiangXu/A-mem-sys
• mem0ai/mem0

Best for: Research Scientist, NLP Engineer, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• NextMem: Towards Latent Factual Memory for LLM-based Agents — NextMem uses an autoregressive autoencoder and two-stage training to create efficient, reconstructible latent factual memory for LLM agents.
• Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agents — MAGE manages agent memory as a hierarchical execution state tree, improving long-horizon task performance and efficiency.
• SWE-MeM: Learning Adaptive Memory Management for Long-Horizon Coding Agents — Long-horizon coding agents can achieve superior performance and efficiency through adaptive, proactive memory management.
• Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning — Training LLMs with end-to-end RL for long-lifecycle agents fosters continuous learning and cross-domain generalization.
• Rosetta Memory: Adaptive Memory for Cross-LLM Agents — Rosetta Memory enables cross-LLM agents to share and adapt memory effectively, shifting from LLM-centric to memory-centric design.
• A Practical Guide to Memory for Autonomous LLM Agents — Memory architecture, particularly its management, is more critical to LLM agent performance than model choice.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.