Agent Memory Engineering

2026-05-01 · Source: Nicolas Bustamante · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Robotics & Autonomous Systems · Depth: Advanced, extended

Summary

Agent memory engineering reveals that large language models (LLMs) are post-trained on their specific harness's memory layer, making memory transfer between agents like Claude Code and Codex challenging. The article analyzes three production implementations: Hermes (Nous Research), Codex CLI (OpenAI), and Claude Code (Anthropic). It finds that simple markdown files and filesystem tools, rather than complex vector databases or knowledge graphs, are winning in production. Key architectural differences include when memory is written (synchronous vs. deferred), how much context is always loaded (snapshot vs. index-only vs. full index), and eviction policies (hard char cap, usage decay, or verification reminders). The prefix cache problem is a critical constraint, driving designs that freeze system prompt snapshots or inject dynamic data into user messages to maintain cost efficiency.

Key takeaway

For AI Architects designing agent systems, prioritize simple, text-file-based memory solutions over complex vector databases. Your design should freeze system prompt snapshots at session start or inject dynamic memory into user messages to ensure cost-effective long sessions. Crucially, implement a "no-op" default for memory writes and integrate dynamic age verification on every memory read to maintain accuracy and prevent silent failures, especially in evolving codebases.

Key insights

LLM memory is fused with its harness during post-training, hindering direct transfer.

Principles

• Simple markdown files and filesystem tools win for agent memory.
• Dynamic memory must avoid the system prompt to preserve cache.
• Default to no-op for memory writes; make the model justify saving.

Method

Codex uses a two-phase pipeline: a small model extracts raw memories after session idle, then a larger consolidation agent edits a canonical handbook using standard tools.

In practice

• Wrap every memory body read with a dynamic age reminder.
• Implement a signal gate to prevent noise in memory writes.
• Scan memory content for prompt injection threats before saving.

Topics

• Agent Memory
• LLM Architectures
• Prompt Engineering
• Prefix Caching
• Memory Eviction
• Agent Design

Code references

• openai/codex

Best for: AI Engineer, Machine Learning Engineer, AI Architect

Related on AIssential

• How AI Agent Memory Actually Works - And How to Build It — AI agent memory architecture enables stateless LLMs to persist and recall information across sessions using a two-layer, four-type system.
• Designing Agentic Memory in 2026 — Agent memory design requires distinct architectural decisions for storage, retrieval, learning, and security, moving beyond stateless LLMs.
• A Practical Guide to Memory for Autonomous LLM Agents — Memory architecture, particularly its management, is more critical to LLM agent performance than model choice.
• Context Windows Are the New RAM: Memory Architecture for Agentic Systems — Agent context windows are caches, not flat storage, necessitating a four-tier memory hierarchy and active management for scalable, effective AI systems.
• How The Hermes Agent Memory Really Works — Hermes Agent's persistent memory uses a five-layered architecture for dynamic, autonomous AI agent operation.
• rohitg00 / agentmemory — Agentmemory provides persistent, searchable memory for AI coding agents, enhancing efficiency and reducing redundant explanations.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Nicolas Bustamante.