MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery

2026-06-06 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, long

Summary

MLEvolve is an LLM-based, self-evolving multi-agent framework designed for automated machine learning algorithm discovery. It addresses limitations in existing MLE agents, such as inter-branch information isolation, memoryless search, and lack of hierarchical control. MLEvolve integrates Progressive Monte Carlo Graph Search (MCGS) for cross-branch information flow and adaptive exploration, Retrospective Memory for accumulating and reusing task-specific experience, and Hierarchical Planning with Adaptive Code Generation to decouple strategy from implementation. Evaluated on MLE-Bench, MLEvolve achieved a 65.3% average medal rate under a 12-hour budget, outperforming proprietary and open-source baselines, including AlphaEvolve, on 75 end-to-end machine learning tasks and 15 mathematical optimization tasks.

Key takeaway

For machine learning engineers developing autonomous agent systems, MLEvolve demonstrates a robust approach to overcoming common limitations in long-horizon algorithm discovery. You should consider integrating graph-based search, dynamic memory mechanisms, and hierarchical planning with adaptive code generation to enhance agent self-evolution, improve solution quality, and manage complex, iterative ML pipeline optimization tasks more effectively.

Key insights

MLEvolve enables LLM agents to self-evolve for long-horizon ML algorithm discovery via graph search, memory, and adaptive code generation.

Principles

• Cross-branch information flow improves search efficiency.
• Accumulated experience guides future planning.
• Decoupling planning from coding enhances control.

Method

MLEvolve uses Progressive MCGS with graph-based expansion and an entropy-inspired schedule, Retrospective Memory for experience, and Hierarchical Planning with adaptive code generation modes (Base, Stepwise, Diff).

In practice

• Implement graph-based search for complex problem spaces.
• Integrate dynamic memory for iterative task learning.
• Use adaptive code generation for controlled modifications.

Topics

• LLM Agents
• Automated Machine Learning
• Monte Carlo Tree Search
• Graph-based Search
• Retrospective Memory
• Code Generation

Code references

• InternScience/MLEvolve

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery — MLEvolve enables LLM agents to self-evolve for automated machine learning algorithm discovery by integrating advanced search and memory.
• MobEvolve: An Agentic Self-Evolving Heuristic System for Interpretable Human Mobility Generation — MobEvolve uses an LLM agent to self-evolve heuristic rules for human mobility generation, achieving superior performance and interpretability.
• Stop coding AI: Use Runtime Topological Self-Assembly (UC, DeepMind) — LLMs can optimize discrete symbolic graphs to discover new algorithms and dynamically architect multi-agent systems.
• From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms — LLM agent memory evolves through stages of storage, reflection, and experience, driven by consistency, dynamic environments, and continual learning.
• Autoresearch in the Wild: A Survey of Real-World Applications — Autoresearch leverages LLM agents to autonomously optimize code and configurations across diverse domains by iteratively modifying, running, and evaluating changes.
• Papers You Should Know About — AI research is advancing LLM efficiency and agent autonomy through novel architectures, self-improvement mechanisms, and real-world applications.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.