Learning to Play Two-Player Perfect-Information Games without Knowledge

2025-12-31 · Source: JMLR · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Gaming & Interactive Media · Depth: Expert, quick

Summary

Quentin Cohen-Solal's 2026 work introduces Athénan, a unified algorithm designed for learning game state evaluation functions in two-player perfect-information games using reinforcement learning. Athénan integrates five novel techniques: generalizing tree bootstrapping to non-linear function approximation, modifying Unbounded Best-First Minimax, replacing binary game outcomes with richer reinforcement signals, proposing a state resolution completion mechanism, and introducing an ordinal action-selection distribution. Experimental results demonstrate that each technique significantly enhances playing strength. Athénan consistently outperforms ExIt, a leading self-play reinforcement learning method without prior knowledge. Furthermore, Athénan achieves state-of-the-art results on Hex, Othello, and Arimaa, and on the single-player game Morpion Solitaire, all without relying on domain-specific knowledge or handcrafted heuristics.

Key takeaway

For Machine Learning Engineers developing game AI, Athénan offers a robust framework to achieve state-of-the-art performance without relying on extensive domain-specific knowledge. You should consider integrating its techniques, such as generalized tree bootstrapping and richer reinforcement signals, into your reinforcement learning pipelines. This approach can significantly reduce development time for new game environments by eliminating the need for handcrafted heuristics, allowing you to focus on core algorithmic improvements.

Key insights

Athénan combines novel RL techniques to achieve state-of-the-art performance in perfect-information games without domain knowledge.

Principles

• Richer reinforcement signals improve learning.
• Generalizing tree bootstrapping enhances state value learning.
• Novel action-selection distributions boost strength.

Method

Athénan integrates generalized tree bootstrapping, modified Unbounded Best-First Minimax, richer reinforcement signals, a state resolution completion mechanism, and an ordinal action-selection distribution.

In practice

• Apply Athénan to perfect-information games.
• Use richer signals beyond binary outcomes.
• Explore ordinal action-selection for game AI.

Topics

• Reinforcement Learning
• Game AI
• Perfect-Information Games
• Athénan Algorithm
• Tree Bootstrapping
• Minimax Search

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• In game theory, generalists sometimes win out over specialists — General-purpose policy gradient methods can surprisingly outperform specialized game-theoretic algorithms in imperfect-information games.
• [P] Vibecoded on a home PC: building a ~2700 Elo browser-playable neural chess engine with a Karpathy-inspired AI-assisted research loop — An AI-assisted research loop can rapidly develop high-performance neural chess engines on consumer hardware.
• Model-Free Learning — Model-free reinforcement learning enables agents to learn from experience without explicit knowledge of environment dynamics.
• TRN-R1-Zero: Text-rich Network Reasoning via LLMs with Reinforcement Learning Only — TRN-R1-Zero enables zero-shot reasoning on text-rich networks using reinforcement learning without supervised fine-tuning.
• StarOR: Synergizing Tree Search and Test-Time Reinforcement Learning for Optimization Modeling — StarOR integrates MCTS and Test-Time Reinforcement Learning to adaptively refine optimization modeling policies without ground-truth labels.
• Agentic Transformers Provably Learn to Search via Reinforcement Learning — Transformers can provably learn complex tree search behaviors through structured reinforcement learning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by JMLR.