Giving Your AI Agent Less Context Made It Better at Its Job

2026-07-12 · Source: LLM on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Intermediate, short

Summary

A Microsoft Research team demonstrated that reducing context significantly improves AI agent performance and efficiency. Using a GPT-5 agent for hotel expense report itemization in Microsoft Dynamics 365, they found a full-history agent achieved only 71% task completion, consuming 1.48 million tokens and 14.56 hours for 50 tasks. Conversely, an agent retaining only the last five tool interactions plus a compact summary reached 91.6% completion, using 553,000 tokens in under six hours—2.7 times cheaper and more effective. Full-history failures stemmed from stale information, while pruning alone caused premature termination. Combining pruning with a summary resolved both, reducing total failures from 73 to 21 across 50 tasks. These findings, detailed in their June 2026 paper "Less Context, Better Agents: Efficient Context Engineering for Long-Horizon Tool-Using LLM Agents," held across expense categories and with Claude Sonnet 4.5, indicating broad applicability.

Key takeaway

For AI Engineers building long-horizon, tool-using agents, relying solely on larger context windows to improve performance is counterproductive. You should instead implement intelligent context management strategies, such as pruning older, stale tool interactions and integrating a compact, auto-generated summary of past actions. This approach significantly boosts task completion rates and reduces operational costs, preventing your agents from "drowning" in irrelevant historical data and improving their reliability in production.

Key insights

Less context, specifically curated, enhances AI agent performance and cost-efficiency on long-horizon tasks.

Principles

• Full context windows can degrade agent performance due to stale information.
• Effective context management is a memory-management problem, not just context-size.
• Summarization can restore agent's sense of progress after context pruning.

Method

Manage agent context by retaining only the last few tool interactions and generating a compact, auto-generated summary of all older, pruned information to maintain task awareness.

In practice

• Implement context pruning to reduce token usage and processing time.
• Combine pruning with dynamic summarization for long-horizon tasks.
• Prioritize context relevance over sheer volume for tool-using agents.

Topics

• AI Agents
• Context Management
• LLM Performance
• Token Efficiency
• Microsoft Dynamics 365
• Claude Sonnet 4.5

Best for: AI Architect, AI Scientist, Research Scientist, AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Why Bigger Context Windows Make AI Worse — Large context windows degrade LLM performance and increase costs; effective context compaction is crucial for agent reliability.
• Context Window Management for Long-Running Agents: Strategies and Tradeoffs — Managing context windows in long-running AI agents requires specific strategies to overcome bottlenecks and tradeoffs.
• Why More Context Makes Your Agent Dumber and What to Do About It — Nupur Sharma, Qodo — LLM agents struggle with large contexts, planning loops, and task overload; structured multi-agent systems and context optimization are key.
• Context Engineering for AI Agents: Part 2 — Effective AI agent performance hinges on precise context management, minimizing irrelevant information and optimizing tool access.
• Building a Context Pruning Pipeline for Long-Running Agents — Efficiently manage LLM context for long-running agents by dynamically pruning conversation history based on semantic relevance.
• 🥇Top AI Papers of the Week — AI advancements focus on improving reasoning, context management, and personalization for LLMs and agents.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by LLM on Medium.