CAMeR: Keyword-Gated Hybrid Activation for Adaptive Memory Retention in LLM Agents

2026-07-25 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

CAMeR (Context-Activated Memory Reinforcement), a novel framework for LLM agents' memory retention, combines keyword-gated hybrid activation—a joint symbolic (word-level Jaccard) and sub-symbolic (embedding cosine) gating mechanism—with adaptive weight dynamics. CAMeR computes a hybrid similarity score, reinforcing memories exceeding a 0.25 threshold while others undergo controlled decay. The authors introduce CAMeR-Bench, a 76-memory, 100-round benchmark spanning 8 topic clusters, designed to test adaptive retention where existing benchmarks fail. On CAMeR-Bench, CAMeR's keyword gate achieves a 1.6x larger retention gap (0.039 vs. 0.024) between high-frequency and never-referenced memories compared to embedding-only gating. Its top-5 retrieval saves 83.2% tokens (39k vs. 231k cumulative) versus full-context approaches and improves retrieval precision. Ablation studies confirm the keyword gate, not learnable decay, is the primary performance driver.

Key takeaway

For Machine Learning Engineers developing LLM agents for long-running applications, you should integrate hybrid symbolic-neural memory gating to improve retention efficiency. By combining keyword Jaccard similarity with embedding cosine, you can significantly reduce false-positive memory activations, leading to 83.2% token savings and enhanced retrieval precision compared to embedding-only or full-context approaches. Consider adopting a fixed multiplicative decay for memory weights, as it proved more effective than learned decay at current scales.

Key insights

Combining symbolic keyword overlap with sub-symbolic embedding similarity significantly improves LLM agent memory retention by filtering false positives.

Principles

• Hybrid symbolic-neural gating enhances memory differentiation in LLM agents.
• Keyword sparsity effectively suppresses false-positive memory activations.
• Fixed multiplicative decay with additive reinforcement can outperform learned decay at small scales.

Method

CAMeR computes a hybrid score (0.6 embedding cosine + 0.4 keyword Jaccard) for memory-query pairs. Memories exceeding a 0.25 threshold are reinforced (+0.2), while all memories undergo multiplicative decay (x0.99).

In practice

• Implement keyword-gated hybrid activation to reduce false-positive memory activations.
• Use a fixed multiplicative decay rate (e.g., 0.99) for memory weights in LLM agents.
• Develop custom benchmarks like CAMeR-Bench to evaluate adaptive memory retention.

Topics

• LLM Agents
• Memory Systems
• Hybrid Gating
• Keyword Extraction
• Adaptive Retention
• CAMeR-Bench

Code references

• mem0ai/mem0
• MaartenGr/KeyBERT

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Multi-Head Recurrent Memory Agents — The core problem is memory retention in long-context LLMs; MHM improves this by architecturally shielding memory heads.
• SCM: Sleep-Consolidated Memory with Algorithmic Forgetting for Large Language Models — SCM introduces a brain-inspired memory architecture for LLMs, combining consolidation and forgetting for efficient, accurate recall.
• MemRefine: LLM-Guided Compression for Long-Term Agent Memory — LLM-guided memory compression, prioritizing factual content over surface similarity, effectively manages agent memory within fixed budgets.
• Rosetta Memory: Adaptive Memory for Cross-LLM Agents — Rosetta Memory enables cross-LLM agents to share and adapt memory effectively, shifting from LLM-centric to memory-centric design.
• Gated DeltaNet-2: Better Memory Editing for Linear Attention — The latest LLM advancements focus on architectural refinements and hybrid decoding for improved efficiency and performance.
• Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting — LoRA fine-tuning can cause significant forgetting, which a new output space regularizer effectively mitigates without replay data.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.