Remember, Don't Re-read: Stateful ReAct Agents for Token-Efficient Autonomous Experimentation

2026-06-12 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Intermediate, quick

Summary

A new approach reformulates the autoresearch pattern for autonomous experimentation, which traditionally uses large language models (LLMs) to iteratively modify code for metric optimization. The conventional stateless design incurs an O(n) token cost per iteration and O(n^2) total due to reconstructing context from scratch. This work introduces a stateful ReAct agent, implemented with LangGraph, that utilizes typed persistent state and a tool-calling interface to carry experimental history across iterations, achieving an O(1) token cost per iteration. Evaluated on hyperparameter tuning (15 iterations) and code performance optimization (40 iterations), the stateful agent demonstrated significant token reductions: 90% fewer tokens (2,492 vs. 24,465) for hyperparameter tuning and 52% fewer tokens (627K vs. 1,275K) for code optimization, all while maintaining comparable optimization quality. This structural token reduction stems from avoiding full history re-reads.

Key takeaway

For Machine Learning Engineers running autonomous experimentation workflows with LLMs, adopting stateful ReAct agents can drastically cut operational costs. You can achieve up to 90% token reduction in tasks like hyperparameter tuning and 52% in code optimization by implementing persistent state via a tool-calling interface. This approach ensures your LLM agents operate with fixed-size conversation windows, making iterative development far more efficient and economical.

Key insights

Stateful ReAct agents significantly reduce token costs in autonomous LLM experimentation by maintaining context across iterations.

Principles

• Stateless LLM agents incur quadratic token costs.
• Persistent state reduces token cost to O(1) per iteration.
• Tool-calling interfaces enable stateful agent design.

Method

Reformulate autoresearch as a stateful ReAct agent using LangGraph. Employ typed persistent state to carry experimental history via a tool-calling interface, avoiding full history re-reads.

In practice

• Implement stateful agents for hyperparameter tuning.
• Apply stateful agents to code performance optimization.
• Use LangGraph for managing agent state and tools.

Topics

• Stateful Agents
• ReAct Agents
• LangGraph
• Token Efficiency
• Autonomous Experimentation
• Hyperparameter Tuning

Best for: AI Architect, Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• Researchers let AI Agents Optimize LLM Reasoning and Cut Tokens by 70% — AI agents can autonomously optimize LLM reasoning strategies, significantly reducing token usage while maintaining accuracy.
• State-Grounded Multi-Agent Synthetic Data Generation for Tool-Augmented LLMs — StateGen uses a state-grounded multi-agent LLM loop to generate high-quality synthetic data, eliminating tool-call hallucinations.
• Tool Attention Is All You Need: Dynamic Tool Gating and Lazy Schema Loading for Eliminating the MCP/Tools Tax in Scalable Agentic Workflows — Protocol-level efficiency, not raw context length, is a binding constraint for scalable agentic systems.
• Introducing stateful MCP client capabilities on Amazon Bedrock AgentCore Runtime — Stateful MCP on Amazon Bedrock AgentCore Runtime enables interactive, multi-turn AI agent workflows.
• The Infrastructure Behind Making Local LLM Agents Actually Useful — Local LLM agents require deliberate infrastructure for performance and long-session stability beyond basic model serving.
• Single Agent vs Multi-Agent: When to Build a Multi-Agent System — AI agents leverage LLMs, tools, and memory, often using the ReAct pattern, to perform complex tasks through reasoning and action.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.