Organize then Retrieve: Hierarchical Memory Navigation for Efficient Agents

2026-06-10 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

HORMA, a Hierarchical Organize-and-Retrieve Memory Agent, addresses large language model (LLM) agents' challenges with long-horizon tasks, which stem from statelessness, growing input contexts, and resulting degraded reasoning, increased inference costs, and higher latency. Unlike existing lossy compression or similarity-based retrieval methods that often miss temporal structure, HORMA organizes experiences into a file-system-like hierarchy. This structure links summarized entities to raw trajectories, enabling efficient, detailed access. The system operates in two stages: structured memory construction, which refines experience organization by differentiating failure types, and navigation-based retrieval. The latter employs a reinforcement learning-trained agent to traverse the hierarchy, selecting minimal yet sufficient context to reduce latency. Across benchmarks like ALFWorld, LoCoMo, and LongMemEval, HORMA significantly improves task performance under context constraints, using at most 22.17% of baseline token usage in long conversation tasks and demonstrating superior efficiency-performance trade-offs and generalization.

Key takeaway

For Machine Learning Engineers designing LLM agents for complex, long-horizon tasks, traditional context management methods are proving inefficient and costly. You should consider implementing hierarchical memory structures like HORMA's file-system approach. This method significantly reduces token usage by up to 77.83% and improves reasoning quality by enabling efficient, detailed context retrieval, thereby enhancing agent performance and reducing operational latency.

Key insights

HORMA uses a hierarchical, file-system-like memory to enable efficient, detailed context retrieval for LLM agents in long-horizon tasks.

Principles

• Hierarchical memory improves LLM agent efficiency.
• Distinguish failure types for better memory structure.
• Minimal context selection reduces inference latency.

Method

HORMA constructs structured memory by iteratively refining experiences, distinguishing between information-missing and misleading context failures. It then navigates this hierarchy using an RL agent to retrieve minimal, sufficient context.

In practice

• Implement file-system-like memory for agents.
• Train RL agents for context navigation.
• Evaluate memory systems on long conversation tasks.

Topics

• LLM Agents
• Hierarchical Memory
• Context Management
• Reinforcement Learning
• Long-Horizon Tasks
• ALFWorld

Best for: Research Scientist, NLP Engineer, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agents — MAGE manages agent memory as a hierarchical execution state tree, improving long-horizon task performance and efficiency.
• How xMemory cuts token costs and context bloat in AI agents — xMemory uses a hierarchical memory structure and uncertainty-gated retrieval to optimize long-term AI agent coherence and reduce token costs.
• 🥇Top AI Papers of the Week — AI advancements focus on improving reasoning, context management, and personalization for LLMs and agents.
• OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory — Visual encoding of agent trajectories enables long-horizon memory with reduced token costs and hallucination.
• TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management — TokenMizer uses a knowledge graph to structure LLM session history, significantly reducing context tokens and improving recall.
• Context Memorization for Efficient Long Context Generation — Attention-state memory externalizes LLM prefixes into a lookup-based memory for efficient, training-free long-context generation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.