ATOM-Bench: A Real-World Benchmark for Atomic Skills and Compositional Generalization in Manipulation Policies

2026-06-15 · Source: Artificial Intelligence · Field: Technology & Digital — Robotics & Autonomous Systems, Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

ATOM-Bench is a new real-world benchmark designed to diagnose generalization capabilities in generalist robotic manipulation policies by evaluating both atomic skills and compositional generalization. This benchmark factorizes tabletop manipulation into motor atoms and instruction atoms, featuring 30 atomic tasks and 24 held-out compositional tasks across paired single-arm and dual-arm robot tracks. Researchers collected 3,000 human demonstrations for atomic fine-tuning and performed 2,700 physical rollouts on five representative manipulation policies. ATOM-Bench introduces Atomic Score (AS) and Compositional Failure Share (CFS) to differentiate failure causes. Initial evaluations reveal that while current policies can acquire simple instruction-grounding skills, they struggle with fine-grained motor atoms, counting, and logical filtering. Crucially, strong performance on atomic tasks does not consistently translate to success on held-out compositional tasks.

Key takeaway

For Robotics Engineers developing generalist manipulation policies, you should prioritize rigorous evaluation beyond atomic task success. Your policies must demonstrate robust compositional generalization, as strong atomic performance does not reliably transfer. Utilize benchmarks like ATOM-Bench to diagnose whether failures stem from weak motor execution, poor instruction grounding, or limited compositional reuse, guiding your model improvements effectively.

Key insights

Current robotic manipulation policies struggle with fine motor skills and compositional generalization, even when atomic skills are strong.

Principles

• Manipulation can be factorized into motor and instruction atoms.
• Atomic skill proficiency does not guarantee compositional generalization.
• Failures can be diagnosed by motor, instruction, or compositional limits.

Method

ATOM-Bench evaluates policies by fine-tuning on 30 atomic tasks and testing on 24 held-out compositional tasks across single/dual-arm tracks, using Atomic Score (AS) and Compositional Failure Share (CFS) to diagnose failures.

In practice

• Benchmark generalist manipulation policies with ATOM-Bench.
• Diagnose policy failures in motor execution or instruction grounding.
• Fine-tune atomic skills using human demonstration data.

Topics

• Robotic Manipulation
• Generalization Benchmarks
• Atomic Skills
• Compositional Generalization
• Robot Learning
• Policy Evaluation

Best for: Research Scientist, AI Scientist, Robotics Engineer, Machine Learning Engineer

Related on AIssential

• Physical Intelligence shows robot model with LLM-like generalization, flaws included — π0.7 is a robot foundation model exhibiting compositional generalization by recombining learned skills.
• DuoBench: A Reproducible Benchmark for Bimanual Manipulation in Simulation and the Real World — DuoBench offers a reproducible benchmark for bimanual robot manipulation, revealing current policy limitations in coordination and sim-to-real transfer.
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies — RoboLab offers a high-fidelity simulation benchmark to assess robotic policy generalization and sensitivity to external factors.
• Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation — GTP-FA improves robotic manipulation by attributing failures to optimize distinct grasping and planning stages.
• Factored Diffusion Policies:Compositionally Generalized Robot Control with a Single Score Network — Factored Diffusion Policies use a single diffusion network with additive score decomposition for compositionally generalized robot control.
• Is Diversity All You Need for Scalable Robotic Manipulation? — Not all data diversity benefits robotic manipulation equally; task diversity and debiased expert data are key.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.