NextMem: Towards Latent Factual Memory for LLM-based Agents

2015-08-23 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

NextMem introduces a latent factual memory framework for LLM-based agents, addressing limitations of existing textual and parametric memory methods. It utilizes an autoregressive autoencoder to efficiently construct and accurately reconstruct latent memory. The framework employs a two-stage training process: autoregressive reconstruction alignment and progressive latent substitution, further incorporating 4-bit NormalFloat (NF4) quantization to reduce storage overhead. Extensive experiments demonstrate NextMem's superior performance in factual reconstruction, contextual generation, and dense passage retrieval across datasets like SQuAD, HotpotQA, and RACE. The model, built on Qwen3-8B, generates 15 latent tokens and shows robustness to noise and effective semantic assignment, offering a scalable solution for memory in LLM-based agents.

Key takeaway

Research Scientists developing LLM-based agents should consider NextMem's latent factual memory framework to overcome context length and catastrophic forgetting issues. Its ability to efficiently store, reconstruct, and retrieve information via compact latent representations, coupled with 4-bit quantization, offers a scalable and robust alternative to traditional textual or parametric memory, potentially improving agent performance and reducing operational costs.

Key insights

NextMem uses an autoregressive autoencoder and two-stage training to create efficient, reconstructible latent factual memory for LLM agents.

Principles

• Factual memory requires lossless preservation.
• Latent representations can unify memory storage and retrieval.
• Progressive training enhances latent representation learning.

Method

NextMem employs an autoregressive autoencoder with shared encoder/decoder weights, trained in two stages: autoregressive reconstruction alignment and progressive latent substitution. NF4 quantization is applied for storage reduction.

In practice

• Use NF4 quantization for latent memory compression.
• Implement progressive latent substitution for robust training.
• Employ a special token like "[SoD]" for transformation initiation.

Topics

• LLM Agents
• Latent Memory
• Autoregressive Autoencoders
• Memory Quantization
• Factual Memory

Code references

• nuster1128/NextMem
• snap-research/LoCoMo
• deepseek-ai/DeepSeek-OCR

Best for: Research Scientist, AI Researcher, AI Scientist, Deep Learning Engineer

Related on AIssential

• MemRefine: LLM-Guided Compression for Long-Term Agent Memory — MemRefine uses an LLM judge to compress agent memory, preserving factual content within fixed storage budgets.
• AtomMem: Building Simple and Effective Memory System for LLM Agents via Atomic Facts — AtomMem enhances LLM agent memory by structuring interactions into atomic facts, hierarchical events, and temporal profiles for efficient, associative retrieval.
• Drawing on Memory: Dual-Trace Encoding Improves Cross-Session Recall in LLM Agents — Dual-trace memory encoding enhances LLM agent recall by adding contextual "scene traces" to factual memories.
• New agentic memory framework uses 118K tokens per query. LangMem burns through 3.26M. — MRAgent dynamically reconstructs memory during reasoning, significantly reducing token use and improving long-horizon task performance.
• RaMem: Contextual Reinstatement for Long-term Agentic Memory — RaMem prevents "context collapse" in LLM agent memory by reinstating original episodic context to validate retrieved fragments.
• Rethinking Personalization for the Agent Era, Semantic Recall for Vector Search, and More! — LLMs are increasingly central to advancing information retrieval and recommender system capabilities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.