EvoForest: A Novel Machine-Learning Paradigm via Open-Ended Evolution of Computational Graphs

2026-03-24 · Source: cs.NE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

EvoForest is a novel hybrid neuro-symbolic system designed for open-ended evolution of computational graphs in machine learning, particularly for structured prediction problems where discovering appropriate computations is critical. Unlike traditional methods focused on parameter optimization, EvoForest jointly evolves reusable computational structures, callable function families, and trainable low-dimensional continuous components within a shared directed acyclic graph (DAG). It directly evaluates candidate graph configurations against non-differentiable cross-validation targets, using a lightweight Ridge-based readout. The system incorporates a diagnostic feedback loop, converting evaluator internals into structured feedback that guides future LLM-driven mutation proposals. In the 2025 ADIA Lab Structural Break Challenge, EvoForest achieved a 94.13% ROC-AUC after 600 evolution steps, surpassing the publicly reported winning score of 90.14% under the same protocol. This framework exemplifies a "search-first" paradigm, emphasizing explicit discovery, reuse, refinement, and selection of useful computations.

Key takeaway

For AI Engineers and Research Scientists tackling complex structured prediction problems with non-differentiable objectives, EvoForest offers a compelling alternative to traditional parameter-centric ML. Its "search-first" approach, which explicitly discovers and refines computational structures, can yield superior performance and even transferable algorithmic knowledge, as demonstrated by the Multi-Space Random Features (MSRF) discovery. You should explore adopting evolutionary computation frameworks that integrate diagnostic feedback and LLM-guided search to uncover novel, high-performing computational motifs.

Key insights

EvoForest evolves computational graphs and functions, guided by diagnostics, to discover optimal computations for structured prediction.

Principles

• Learning as computation search, not just parameter fitting.
• Diagnostic feedback drives LLM-guided evolutionary mutations.
• Multi-alternative DAGs enable combinatorial search and reuse.

Method

EvoForest uses an asynchronous island model with LLM-guided "scientist" and "engineer" stages to propose and synthesize graph mutations, refining continuous parameters via gradient descent and scoring configurations with a Ridge model.

In practice

• Apply EvoForest for non-differentiable, cross-validation-based objectives.
• Consider multi-space feature allocation for time-series problems.
• Use diagnostic feedback to guide feature engineering.

Topics

• EvoForest
• Computational Graph Evolution
• Neuro-Symbolic AI
• Search-First Machine Learning
• ADIA Lab Structural Break Challenge

Code references

• karpathy/autoresearch

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Director of AI/ML

Related on AIssential

• MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery — MLEvolve enables LLM agents to self-evolve for automated machine learning algorithm discovery by integrating advanced search and memory.
• EvoCSFL: Surrogate-Assisted Evolutionary Client Selection for Efficient and Robust Federated Learning — EvoCSFL uses a surrogate-assisted evolutionary algorithm to optimize client selection in federated learning for improved efficiency and robustness.
• EvoSelect: Data-Efficient LLM Evolution for Targeted Task Adaptation — EvoSelect efficiently adapts LLMs by selecting diverse, task-aligned synthetic data using optimal transport and diversity regularization.
• EvoPool: Evolutionary Programmatic Annotation for Label-Efficient Specialized Supervision — EvoPool uses evolutionary multi-agent programmatic annotation to generate high-quality, cost-effective labels for specialized tasks, outperforming LLMs.
• [Paper] Stringological sequence prediction I — Novel stringology-based algorithms offer efficient, mistake-bounded deterministic sequence prediction, bridging theory and practical AI.
• Associative Constructive Evolution: Enhancing Metaheuristics through Hebbian-Learned Generative Guidance — ACE enhances metaheuristics by learning and reusing successful operation patterns through a generative automaton.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.NE updates on arXiv.org.