Multi-action Tangled Program Graphs for Multi-task Reinforcement Learning with Continuous Control

2026-04-29 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, short

Summary

Researchers from IETR introduce Multi-Action Tangled Program Graphs (MATPG), a Genetic Programming (GP) algorithm designed for Multi-Task Reinforcement Learning (MTRL) in continuous control environments. MATPG extends the Tangled Program Graph (TPG) algorithm, originally for discrete MTRL, by aggregating MAPLE agents and creating a control flow to activate them. Initially, MATPG showed performance similar to MAPLE in single-task RL. This work presents a new benchmark based on the MuJoCo Half Cheetah from Gymnasium, featuring five distinct, randomly positioned obstacles, each requiring a unique behavior. Experiments on this benchmark demonstrate MATPG's superior performance in multi-task continuous control when combined with lexicase selection. The study also highlights the full interpretability of the evolved decision flow graph.

Key takeaway

For research scientists developing multi-task reinforcement learning solutions, MATPG offers a robust and interpretable genetic programming approach for continuous control. You should consider integrating MATPG, especially with lexicase selection, into your experimental designs to achieve superior performance and gain clear insights into decision-making processes, particularly in complex environments like the MuJoCo Half Cheetah benchmark.

Key insights

MATPG combines genetic programming and agent aggregation for interpretable, superior multi-task continuous reinforcement learning.

Principles

• Genetic Programming excels in MTRL.
• Lexicase selection improves MATPG performance.
• Evolved graphs offer full interpretability.

Method

MATPG aggregates MAPLE agents and establishes a control flow to activate them, enabling a single model to learn multiple behaviors in continuous MTRL environments.

In practice

• Apply MATPG to continuous MTRL tasks.
• Use lexicase selection for enhanced results.
• Leverage MATPG for interpretable RL models.

Topics

• Multi-Task Reinforcement Learning
• Continuous Control
• Tangled Program Graphs
• Multi-Action TPG
• Genetic Programming

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Solvita: Enhancing Large Language Models for Competitive Programming via Agentic Evolution — Solvita enhances LLMs for competitive programming through a multi-agent, reinforcement learning framework with continuous, experience-driven knowledge networks.
• A Model-Driven Approach for Developing Families of Reinforcement Learning Environments — A model-driven approach using a hybrid genetic algorithm automates the creation of diverse reinforcement learning environment families.
• Provable Offline Reinforcement Learning for Structured Cyclic MDPs — CycleFQI offers a modular, theoretically robust offline RL solution for cyclic MDPs with heterogeneous stages.
• RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agents — RODS dynamically synthesizes data by detecting capability boundaries, resolving sample depletion in multi-turn tool-use RL.
• Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning — Preserving diverse policy neighborhoods in a stable latent space enhances continual reinforcement learning plasticity.
• Coordination Graphs for Constrained Multi-Agent Reinforcement Learning — The CG-CMARL framework efficiently scales multi-agent reinforcement learning by combining coordination graphs and Lagrangian duality for constraint handling.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.