DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Memory QA

2026-05-21 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, quick

Summary

DeferMem is a novel long-term memory framework designed to enhance large language model (LLM) agents' performance in question answering tasks involving extensive conversational histories. Addressing the challenge of scattered and irrelevant evidence, DeferMem decouples memory processing into high-recall candidate retrieval and query-conditioned evidence distillation. It employs a lightweight segment-link structure to organize raw history and retrieve broad candidates at query time. A key component is its memory distiller, trained using DistillPO, a reinforcement learning algorithm. DistillPO formulates post-retrieval evidence distillation as a structured action involving message selection and evidence rewriting, optimized via a decomposed-and-gated reward pipeline. On benchmarks like LoCoMo and LongMemEval-S, DeferMem significantly surpasses strong baselines, achieving the highest QA accuracy with the fastest runtime and zero commercial-API token cost for memory operations.

Key takeaway

For Machine Learning Engineers developing LLM agents that require robust long-term memory, DeferMem offers a compelling alternative to traditional memory systems. You should consider its decoupled retrieval and RL-driven distillation approach to improve QA accuracy and system efficiency. This framework eliminates commercial-API token costs for memory operations, providing a cost-effective solution for managing extensive conversational histories and reducing downstream denoising efforts.

Key insights

DeferMem improves LLM long-term memory QA by decoupling retrieval from query-conditioned evidence distillation using RL.

Principles

• Decouple high-recall retrieval from query-conditioned distillation.
• Optimize evidence distillation with structured RL actions.
• Gate reward components from validity to quality checks.

Method

DeferMem organizes history with a segment-link structure for broad candidate retrieval. It then applies a DistillPO-trained memory distiller for message selection and evidence rewriting, optimizing with a decomposed-and-gated reward pipeline.

In practice

• Improve LLM QA over long conversational histories.
• Reduce commercial-API token costs for memory.
• Enhance QA accuracy and system runtime efficiency.

Topics

• Long-Term Memory QA
• Large Language Models
• Reinforcement Learning
• Evidence Distillation
• LLM Agents
• Information Retrieval

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Memory QA — Query-time evidence distillation via reinforcement learning significantly improves LLM agent long-term memory QA efficiency and accuracy.
• MemTrace: Probing What Final Accuracy Misses in Long-Term Memory — MemTrace evaluates LLM long-term memory by probing knowledge points across controlled dimensions, revealing hidden failure modes beyond aggregated accuracy.
• ABBEL: Learning Natural-Language Belief States for Memory-Efficient Interaction — LLM agents can use natural language belief states to compress interaction history, improving memory efficiency and interpretability.
• Phase Transitions in Agent Memory: Recurrent Memory — Long-running LLM agents benefit from density-driven memory consolidation, triggering processing only upon sustained semantic recurrence.
• MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA — MARDoc uses a structured, dynamically updated memory to overcome context dilution in multimodal long-document QA agents.
• MemRerank: Preference Memory for Personalized Product Reranking — Distilling user purchase history into concise, query-independent preference memory significantly boosts LLM-based product reranking accuracy.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.