MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery

2026-06-04 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, quick

Summary

MLEvolve is an LLM-based self-evolving multi-agent framework designed for end-to-end machine learning algorithm discovery, addressing limitations in existing MLE agents such as information isolation and memoryless search. It extends tree search to Progressive MCGS, enabling cross-branch information flow via graph-based reference edges and shifting search from exploration to exploitation using an entropy-inspired schedule. The framework incorporates Retrospective Memory, which integrates a cold-start domain knowledge base with a dynamic global memory for task-specific experience retrieval and reuse. For stable long-horizon iteration, MLEvolve decouples strategic planning from code generation through adaptive coding modes. Evaluated on MLE-Bench, MLEvolve achieves leading performance in average medal rate and valid submission rate within a 12-hour budget, half the standard runtime. It also surpasses specialized methods like AlphaEvolve on mathematical algorithm optimization tasks, demonstrating strong cross-domain generalization.

Key takeaway

For Machine Learning Engineers tasked with automating algorithm discovery or optimizing long-horizon ML engineering tasks, MLEvolve presents a significant advancement. You should consider exploring frameworks that integrate progressive tree search, dynamic memory, and decoupled planning to overcome limitations of existing agents. This approach can yield leading performance and strong cross-domain generalization, potentially halving the time budget for complex algorithm development.

Key insights

MLEvolve enables LLM agents to self-evolve for automated machine learning algorithm discovery by integrating advanced search and memory.

Principles

• Enable cross-branch information flow in tree search.
• Progressively shift search from exploration to exploitation.
• Evolve agents with accumulated task-specific experience.

Method

MLEvolve employs Progressive MCGS for search, Retrospective Memory for experience management, and adaptive coding modes for stable long-horizon iteration in algorithm discovery.

In practice

• Achieve leading ML algorithm discovery performance.
• Generalize performance across diverse optimization tasks.

Topics

• LLM Agents
• Automated Machine Learning
• Algorithm Discovery
• Multi-agent Systems
• Progressive MCGS
• Retrospective Memory

Code references

• InternScience/MLEvolve

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• ML-Evolve— A Self-Evolving Agent System for Algorithm Optimization — ML-Evolve automates ML algorithm optimization using a multi-agent, multi-island evolutionary system with explicit research grounding and compute awareness.
• MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery — MLEvolve enables LLM agents to self-evolve for long-horizon ML algorithm discovery via graph search, memory, and adaptive code generation.
• MetaEvo: A Meta-Optimization Framework for Experience-Driven Agent Evolution — MetaEvo enables LLM agents to continually improve reasoning by optimizing how they learn principles from experience, not just what they store.
• Import AI 439: AI kernels; decentralized training; and universal representations — AI systems can automate and accelerate complex software optimization tasks, yielding significant performance gains.
• FunctionEvolve: Structure-Guided Symbolic Regression with LLMs — Structure-guided symbolic regression with LLMs significantly improves exact form recovery by using expression trees.
• EvoForest: A Novel Machine-Learning Paradigm via Open-Ended Evolution of Computational Graphs — EvoForest evolves computational graphs and functions, guided by diagnostics, to discover optimal computations for structured prediction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.