Your AI Agent Already Forgot Half of What You Told It

2026-05-28 · Source: AI & ML – Radar · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, extended

Summary

Andrew Stellman's May 28, 2026 article, "Your AI Agent Already Forgot Half of What You Told It," addresses the common problem of AI agents and chatbots losing context mid-workflow due to "context compaction." This occurs when an AI's fixed-size context window fills, causing it to truncate, compress, or inconsistently drop older information without notification. Stellman, drawing parallels to human team communication challenges from his 2005 book "Applied Software Project Management," proposes four techniques to externalize critical information. These include splitting discovery from documentation by writing observations to files like CONTRACTS.md, using AI-generated handoff documents for new sessions, providing acceptance criteria instead of step-by-step procedures, and employing spec documents as a single source of truth to bridge multiple AI tools. These methods, developed for projects like Octobatch and Quality Playbook, are shown to be effective for both AI skill development and general chatbot interactions, including managing the article series itself.

Key takeaway

For AI Engineers and Prompt Engineers managing complex AI workflows, proactively externalizing context is crucial to prevent silent information loss. You should implement strategies like splitting discovery from documentation, generating handoff documents for session continuity, and defining clear acceptance criteria instead of step-by-step procedures. Additionally, use shared spec documents as a single source of truth across different AI tools to maintain consistent design intent and avoid costly re-explanation.

Key insights

AI context loss is mitigated by externalizing information to persistent files.

Principles

• AI context windows have fixed sizes; information is lost when full.
• Split complex AI tasks into discovery and documentation phases.
• Define "done" with acceptance criteria, not procedural steps.

Method

Externalize AI's working memory by writing critical information to persistent files, preventing context compaction loss. This involves splitting tasks, creating handoff documents, defining acceptance criteria, and using shared spec documents.

In practice

• Write AI observations to external files like CONTRACTS.md.
• Ask AI to generate handoff documents before ending sessions.
• Define "done" for AI tasks using acceptance criteria.

Topics

• AI Context Management
• Agentic Engineering
• Prompt Engineering
• Context Compaction
• Handoff Documents
• Acceptance Criteria

Code references

• andrewstellman/octobatch
• andrewstellman/quality-playbook

Best for: AI Engineer, Machine Learning Engineer, Prompt Engineer

Related on AIssential

• Why Doesn’t Anyone Teach Developers About Context Management? — Effective AI development hinges on explicit context management, preventing AI amnesia and improving output quality.
• Your AI Agent Isn’t Dumb. It Has ADHD — LLMs forget instructions mid-conversation due to context window attention bias, causing significant performance drops.
• We have built a system that prevents context contamination of AI by using periodic memory updates. — Periodic memory updates and context resets can prevent AI context contamination and improve reliability.
• Context Engineering for AI Agents: Part 2 — Effective AI agent performance hinges on precise context management, minimizing irrelevant information and optimizing tool access.
• The Context Window Trap: Stop Drowning Your AI in Data — Overloading LLM context windows with data degrades performance, increases cost, and erodes engineering craft.
• Why AI Agents Need Structure — AI agent failures stem from structural workflow issues, not model intelligence, requiring context isolation between distinct phases.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI & ML – Radar.