EVOM: Agentic Meta-Evolution of Actor-Critic Architectures for Reinforcement Learning

2026-06-24 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

EVOM is an agentic meta-evolution framework designed to automate the discovery of high-performance actor-critic architectures in reinforcement learning, addressing the challenges of manual design and the extensive training required for each candidate. It employs a bi-level optimization strategy: an inner loop trains network weights using low-fidelity Proximal Policy Optimization (PPO), while an outer loop iteratively refines architecture programs. A crucial component is an LLM-based design agent that drives this meta-evolution, functioning solely as an architecture designer, entirely decoupled from policy execution. Experiments demonstrate that EVOM surpasses manually designed baselines, LLM-guided random search, and MLES, a leading LLM-guided programmatic policy search method, achieving superior performance on Ant-v4 and HalfCheetah-v4. Ablation studies confirm the indispensable roles of both the meta-evolution loop and the LLM Design Agent in achieving its final performance.

Key takeaway

For AI Scientists and Machine Learning Engineers designing actor-critic architectures, manual design is often suboptimal. You should consider integrating LLM-driven agentic meta-evolution frameworks like EVOM into your architecture search workflows. This approach can significantly enhance performance on complex tasks such as Ant-v4 and HalfCheetah-v4 by systematically exploring design spaces more effectively than traditional methods, ultimately leading to superior reinforcement learning policies.

Key insights

EVOM uses an LLM-based agent and bi-level meta-evolution to automate high-performance actor-critic architecture discovery in reinforcement learning.

Principles

• Architecture search can be framed as bi-level optimization.
• Decoupling design agents from execution improves efficiency.
• LLMs can effectively drive meta-evolution for architecture design.

Method

EVOM employs a bi-level optimization: an inner loop trains policy weights via PPO, while an outer loop, driven by an LLM design agent, iteratively refines actor-critic architecture programs.

In practice

• Apply LLMs for automated neural architecture search.
• Use low-fidelity PPO for inner loop weight training.
• Decouple architecture design from policy execution.

Topics

• Reinforcement Learning
• Actor-Critic Architectures
• Neural Architecture Search
• Large Language Models
• Meta-Evolution
• Proximal Policy Optimization

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Large Language Model-Driven Full-Component Evolution of Adaptive Large Neighborhood Search — LLM-driven evolution can automate and enhance complex metaheuristic algorithm design by decoupling and optimizing core components.
• EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments — LLM agents require memory that tracks environmental evolution for robust performance in dynamic real-world settings.
• Solvita: Enhancing Large Language Models for Competitive Programming via Agentic Evolution — Solvita enhances LLMs for competitive programming through a multi-agent, reinforcement learning framework with continuous, experience-driven knowledge networks.
• AI Self EVOLUTION (Meta Harness) — Automating LLM harness engineering via a self-improving coding agent significantly boosts performance and efficiency.
• REFLEX: Reflective Evolution from LLM Experience — REFLEX decouples visual diagnosis from code generation in LLM-guided evolutionary search for transparent, efficient policy discovery.
• EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM Agents — EvoDrive uses LLM-based agentic evolution to generate diverse, safety-critical autonomous driving scenarios by balancing adversariality and realism.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.