Offline Nash Solvers Meet Online Tree Search in Multi-Agent Games on Graphs

2026-07-13 · Source: cs.MA updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

Primitive-Guided Tree Search (PGTS) is a novel hybrid framework designed to solve multi-agent Pursuit-Evasion games (PEGs) on graphs, which are challenging due to exponentially growing state and action spaces. PGTS integrates offline exact Nash equilibrium computation of smaller, tractable sub-games (e.g., 1v1, 2v1) with online tree search. At deployment, it uses these pre-computed optimal sub-game policies and value functions to guide tree expansion and estimate leaf-node values. Experiments on diverse graph topologies, including real-world networks like Atlanta, demonstrate that PGTS significantly outperforms state-of-the-art learning and heuristic baselines, such as MT-PSRO and NSGZero, while maintaining robust performance against various adversaries. The framework achieves strong worst-case utility and expected reward, particularly on complex graphs.

Key takeaway

For AI Scientists and Machine Learning Engineers developing multi-agent systems in adversarial environments, PGTS offers a robust approach to overcome combinatorial complexity. You should consider integrating offline-computed primitive game solutions with online tree search to enhance scalability and maintain strong performance against diverse adversaries. This method provides superior worst-case utility compared to purely learning-based or heuristic baselines, ensuring more reliable decision-making in dynamic, competitive scenarios.

Key insights

PGTS combines offline Nash sub-game solutions with online tree search for robust multi-agent game play.

Principles

• Decompose large multi-agent games into smaller, tractable sub-games.
• Utilize pre-computed sub-game equilibria to guide online tree search.
• Prioritize higher-order primitive interactions (e.g., 2v1) for team coordination.

Method

PGTS first solves primitive sub-team games offline via value iteration. Online, it performs tree search, using primitive policies for guided expansion and primitive value functions for leaf-node estimation, then updates statistics via Regret Matching or Decoupled UCT.

In practice

• Solve 1v1 and 2v1 sub-games offline for reusable policies.
• Employ the Hungarian algorithm for sequential 2v1 then 1v1 agent matching.
• Use Regret Matching or Decoupled UCT for action selection in tree search.

Topics

• Multi-Agent Games
• Pursuit-Evasion Games
• Nash Equilibrium
• Monte Carlo Tree Search
• Game Theory
• Graph-based AI

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• In game theory, generalists sometimes win out over specialists — General-purpose policy gradient methods can surprisingly outperform specialized game-theoretic algorithms in imperfect-information games.
• GATS: Graph-Augmented Tree Search with Layered World Models for Efficient Agent Planning — GATS uses systematic search and a layered world model to eliminate LLM calls, achieving superior, deterministic agent planning.
• Dynamic Resource Allocation for Ensemble Determinization MCTS — Dynamically adjusting determinization trees and simulation budgets significantly enhances MCTS performance in high-uncertainty games.
• MARS$^2$: Scaling Multi-Agent Tree Search via Reinforcement Learning for Code Generation — Multi-agent collaboration within a shared tree search environment enhances RL performance for code generation.
• Parametric Open Source Games — Parametric open-source games model continuous player strategies, revealing how internal parameter access can drive cooperation in game theory.
• PAC Learning in Turn-Based Stochastic Games with Reachability Objectives: A Decentralized Private Approach via Expected Conditional Distance — Decentralized and private PAC learning is now feasible for turn-based stochastic games with reachability objectives.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.MA updates on arXiv.org.