For How Long Should We Be Punching? Learning Action Duration in Fighting Games

2026-05-20 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Gaming & Interactive Media · Depth: Expert, quick

Summary

Reinforcement learning (RL) agents in fast-paced fighting games like Street Fighter II typically operate with hard-coded, fixed decision intervals, which can lead to unrealistic frame-perfect reflexes or reduced responsiveness. A novel decision-making framework proposes that agents learn not only what action to take but also for how long to execute it, enabling dynamic adaptation of responsiveness. This method was implemented using the open-source FightLadder environment, with agents trained against scripted built-in bots and tested across various frame skip configurations. Experiments revealed that learned timing can match the performance of well-chosen fixed frame skips and promotes repeatable action patterns. However, this approach does not inherently guarantee robustness, as agents often performed best with high frame skip values, facilitating exploitative strategies against predictable scripted opponents.

Key takeaway

For Machine Learning Engineers developing RL agents for real-time game environments, consider implementing dynamic action duration learning. This approach allows your agents to adapt responsiveness beyond fixed frame skips, potentially matching optimal fixed configurations. However, ensure robust evaluation against diverse, non-scripted opponents, as learned timing alone does not guarantee generalizability and can lead to exploitative, repetitive behaviors against predictable adversaries.

Key insights

RL agents can learn action duration dynamically, adapting responsiveness in fast-paced fighting games.

Principles

• Fixed decision intervals limit RL agent adaptability.
• Jointly learning action and duration improves dynamic responsiveness.
• High frame skips can facilitate exploitative strategies.

Method

Implement a joint action-duration prediction framework in the FightLadder environment, training agents against scripted bots and testing various frame skip configurations.

In practice

• Integrate action duration learning into RL game agents.
• Test dynamic responsiveness against diverse opponents.
• Consider frame skip impact on learned strategies.

Topics

• Reinforcement Learning
• Fighting Games
• Action Duration Learning
• Frame Skip
• Game AI

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• Finding the Time to Think: Learning Planning Budgets in Real-Time RL — The paper addresses real-time RL by learning state-dependent planning budgets via a lightweight gating policy, outperforming fixed-delay methods.
• Augmenting Game AI with Deep Reinforcement Learning — This paper proposes a framework for training reinforcement learning models to overcome current limitations and accelerate machine learning adoption in game AI.
• Learning to Reason as Action Abstractions with Scalable Mid-Training RL — Mid-training with action abstractions optimizes LLM performance by minimizing RL and approximation errors.
• CEO-Bench: Can Agents Play the Long Game? — LLM agents excel at short tasks but struggle with sustained strategic control, long-horizon planning, and adaptation in complex, dynamic environments.
• Revisiting Action Factorization for Complex Action Spaces — Branching dueling architectures effectively balance compute and performance for complex action spaces, with Auto-Regressive actions achieving peak performance.
• Projecting Latent RL Actions: Towards Generalizable and Scalable Graph Combinatorial Optimization — Projection agents use continuous GNN-based action embeddings for faster, more generalizable graph combinatorial optimization.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.