Your Agents Need a Save Button - Hamza Tahir, ZenML

2026-07-18 · Source: AI Engineer · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, long

Summary

AI agent executions currently lack persistent state, making debugging and optimization difficult as only read-only traces are available. The concept of a "save button" for agents, similar to document auto-save, is proposed to enable replaying past executions and conducting "what-if" scenarios. This allows developers to swap models, mock tools, and analyze outcomes to improve agent cost, speed, and reliability. ZenML's new tool, Kitaru, provides this durable runtime layer, augmenting traces with code execution and environmental snapshots. For instance, DoorDash reduced simulation time to 5 minutes and achieved 90% less hallucinations by replaying customer bots. This approach facilitates closing the evaluation loop by comparing baseline runs with modified replays across cohorts.

Key takeaway

For MLOps Engineers optimizing AI agent performance, implementing a durable runtime with state checkpointing is critical. This enables replaying agent executions, testing "what-if" scenarios with different models or tool configurations, and performing robust cohort analysis. Avoid naive model swaps based on single-run metrics; instead, evaluate changes across diverse production cohorts to ensure real value creation and prevent false economies. Consider integrating tools like Kitaru to streamline this iterative improvement process.

Key insights

Persistent state and replay capabilities are crucial for debugging, optimizing, and evaluating AI agents effectively.

Principles

• Agent state checkpointing enables comprehensive "what-if" analysis.
• Cohort analysis provides robust evaluation over single replays.
• Naive model swaps can create false economies if value isn't assessed.

Method

Checkpoint agent runtime state, replay executions with altered models or mocked tools, diff results, and analyze across cohorts to optimize performance.

In practice

• Implement Kitaru for agent state and execution replay.
• Simulate customer bot interactions to reduce errors.
• Automate evaluation loops for continuous agent improvement.

Topics

• AI Agents
• Agent Orchestration
• Runtime Checkpointing
• Model Evaluation
• Kitaru
• What-if Analysis

Best for: MLOps Engineer, Machine Learning Engineer, AI Engineer

Related on AIssential

• Long-running Agents — Long-running agents overcome AI limitations by externalizing state, enabling sustained, multi-session progress on complex tasks.
• Research Closes the Loop. Production Keeps Us In It. — Reflective prompt evolution, where an LLM edits its own prompts, outperforms RL baselines in research.
• Evaluating Agents Beyond the First Prompt — Multi-turn coding benchmarks reveal agents struggle with regressions and evolving specifications, significantly overstating single-turn reliability.
• The Agent Stack - Part 5: Context, Retrieval, and Memory — Context is the runtime-assembled working set, not the model's inherent knowledge.
• Your Agent Gave the Right Answer for the Wrong Reason — and You Have No Idea — Agentic AI systems require deep observability and multi-dimensional evaluation to prevent silent failures and ensure reliable, efficient operation.
• The Real Bottleneck in AI Agents Is Not the Model — AI agent reliability primarily depends on controlling the execution environment's stability, not solely on model intelligence.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Engineer.