PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents

2026-05-19 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computation and Language · Depth: Expert, quick

Summary

PEEK is a novel system designed to enhance large language model (LLM) agents operating with long and recurring external contexts, such as document corpora or code repositories. Unlike existing methods that preserve agent trajectories or raw material access, PEEK focuses on caching and maintaining "orientation knowledge" about the context itself. This knowledge, including context organization, useful entities, and schemas, is stored as a small, constant-sized "context map" within the agent's prompt. PEEK's maintenance policy comprises a Distiller for extracting transferable knowledge, a Cartographer for structured edits, and a priority-based Evictor for token budget enforcement. Benchmarking shows PEEK improves long-context reasoning and information aggregation by 6.3-34.0%, reducing iterations by 93-145 and costs by 1.7-5.8x compared to ACE, a prominent prompt-learning framework. For context learning, it boosts solving rate and rubric accuracy by 6.0-14.0% and 7.8-12.1%, respectively, at 1.4x lower cost than ACE, demonstrating broad applicability across LMs like OpenAI Codex.

Key takeaway

For Machine Learning Engineers deploying LLM agents in environments with recurring long external contexts, you should re-evaluate traditional context management strategies. PEEK demonstrates that caching "orientation knowledge" via a dynamic context map significantly improves agent accuracy and reduces operational costs by 1.7-5.8x. Consider integrating a similar context map approach to enhance your agents' efficiency and performance, especially for tasks involving document corpora or code repositories. This method offers substantial gains over prompt-learning frameworks like ACE.

Key insights

Caching context orientation knowledge improves LLM agent efficiency and accuracy in recurring long-context tasks.

Principles

• Reusable orientation knowledge is crucial for recurring long-context workloads.
• A small, persistent context map can guide LLM agents.
• Programmable cache policies can maintain dynamic context knowledge.

Method

PEEK's cache policy uses a Distiller to extract knowledge, a Cartographer to translate it into structured edits for the context map, and an Evictor to manage a fixed token budget.

In practice

• Implement a context map for LLM agents handling document corpora.
• Use a programmable cache policy to dynamically update agent context.
• Apply PEEK's modules (Distiller, Cartographer, Evictor) for knowledge management.

Topics

• LLM Agents
• Long-Context LLMs
• Context Map
• Cache Policy
• Orientation Knowledge
• OpenAI Codex

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• CAG vs Long Context: How AI Models Use and Remember Information — Long context and Cache Augmented Generation (CAG) offer distinct strategies for LLM external knowledge access, balancing simplicity, cost, and performance.
• Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents — LLM agent memory should be actively reconstructed and dynamically adapted to reasoning context, not statically retrieved.
• Context Engineering: Memory and Temporal Context — Effective context engineering for LLMs relies on managing short-term, long-term, dynamic, and temporal information.
• A Visual Guide to Attention Variants in Modern LLMs — Modern LLM architectures prioritize efficiency and long-context handling through diverse attention mechanisms and hybrid designs.
• FastContext: Training Efficient Repository Explorer for Coding Agents — Dedicated exploration subagents improve LLM coding agent efficiency and accuracy by providing focused context.
• TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management — LLM session history is a structured knowledge artifact, not flat text, enabling efficient context management.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.