Geometric Action Model for Robot Policy Learning

2026-06-15 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

The Geometric Action Model (GAM) is a novel language-conditioned manipulation policy designed to enhance robot policy learning by directly integrating 3D geometric reasoning. Unlike current vision-language-action models (VLAs) and video world-action models (WAMs) that primarily operate on 2D image frames or latent spaces, GAM repurposes a pretrained geometric foundation model (GFM) as a unified substrate for perception, temporal prediction, and action decoding. Its architecture splits the GFM: shallow layers encode observations, while a causal future predictor forecasts future latent tokens based on language, proprioception, and action history. These predicted tokens then pass through the remaining GFM blocks to produce both future geometry and actions. This design minimally modifies the GFM while preserving its rich geometric priors, resulting in a model that is more accurate, robust, faster, and lighter than existing foundation-model-scale baselines across various simulation and real-robot manipulation benchmarks.

Key takeaway

For Robotics Engineers developing manipulation policies, you should consider integrating explicit 3D geometric reasoning to overcome limitations of 2D-based models. Adopting a Geometric Action Model (GAM) approach, which repurposes pretrained geometric foundation models, can yield more accurate, robust, and efficient robot control for contact-rich tasks. This method allows you to leverage existing geometric priors with minimal architectural changes, potentially accelerating development and improving real-world performance.

Key insights

GAM integrates 3D geometric reasoning into robot policy learning by repurposing a pretrained geometric foundation model for perception, prediction, and action.

Principles

• Repurpose GFMs for unified perception and action.
• Explicit 3D geometry improves manipulation policies.
• Minimal architectural changes preserve priors.

Method

GAM splits a pretrained GFM; shallow layers encode observations, a causal predictor forecasts future latent tokens, and remaining GFM blocks decode these into future geometry and actions.

In practice

• Apply GFMs for language-conditioned robot tasks.
• Use split-layer architecture for temporal prediction.
• Improve contact-rich manipulation accuracy.

Topics

• Robot Policy Learning
• Geometric Foundation Models
• 3D Geometric Reasoning
• Manipulation Robotics
• Vision-Language-Action Models
• Temporal Prediction

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Robotics Engineer, Machine Learning Engineer

Related on AIssential

• LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies — LaWAM enables efficient, dynamics-aware robot control by using compact latent visual subgoals instead of computationally expensive pixel-level video predictions.
• GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation — Distilling 4D geometry foundation model features into video world models ensures correspondence-consistent generation for robot manipulation.
• ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning — Unified data, action manifold learning, and modular 3D perception enable generalizable robotic manipulation across diverse hardware.
• A Tutorial on World Models and Physical AI — World models, explicit or implicit, are central to intelligent systems for prediction, reasoning, and decision-making in physical AI.
• Sergey Levine - Building LLMs for the Physical World - [Invest Like the Best, EP.465] — The core idea is building general robotic foundation models for universal control of embodied systems.
• RepWAM: World Action Modeling with Representation Visual-Action Tokenizers — RepWAM improves robot control and future prediction by using semantic visual-action tokenization in world action modeling.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.