PoseVLA: Universal Pose Pretraining for Generalizable Vision-Language-Action Policies

2026-01-22 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, long

Summary

Pose-VLA is a novel decoupled paradigm for Vision-Language-Action (VLA) models designed to overcome feature collapse and low training efficiency in robotic manipulation. It separates VLA training into a pre-training phase for universal 3D spatial priors in a camera-centric space and a post-training phase for efficient embodiment alignment. By introducing discrete pose tokens as a universal representation, Pose-VLA integrates spatial grounding from diverse 3D datasets with geometry-level trajectories from robotic demonstrations. The framework uses a two-stage pre-training pipeline, establishing spatial grounding via poses and motion alignment through trajectory supervision. Pose-VLA achieves state-of-the-art results on RoboTwin 2.0 with a 79.5% average success rate and competitive performance on LIBERO at 96.0%. Real-world experiments demonstrate robust generalization across diverse objects using only 100 demonstrations per task, leveraging a pre-training corpus of 1.4M images with 6.5M 3D annotations and 1.55M robotic trajectories.

Key takeaway

For Machine Learning Engineers developing generalizable robotic manipulation policies, you should consider adopting a decoupled VLA training approach. By pre-training your Vision-Language Model with universal 3D pose tokens and camera-centric spatial priors, you can significantly reduce the data needed for embodiment alignment. This strategy enables robust generalization across diverse objects with as few as 100 demonstrations per task, mitigating representation collapse and improving policy learning efficiency.

Key insights

Decoupling VLA training with universal pose tokens enables robust 3D spatial grounding and efficient robotic policy learning.

Principles

• Decouple perception from embodiment-specific actions.
• Use discrete pose tokens for universal 3D representation.
• Integrate RGB, depth, and camera intrinsics for 3D awareness.

Method

Pose-VLA employs a two-stage pre-training: spatial foundation on 3D datasets, then pose alignment using multi-view robotic trajectories. Post-training appends a lightweight action expert for robot-specific commands.

In practice

• Pre-train VLMs with 3D pose supervision.
• Project robot actions into camera-centric frame.
• Combine diverse 3D datasets with limited robot demos.

Topics

• Vision-Language-Action Models
• Robotic Manipulation
• 3D Pose Estimation
• Pre-training
• Generalization
• Embodied AI

Best for: Research Scientist, AI Scientist, Robotics Engineer, Machine Learning Engineer

Related on AIssential

• Motion-Focused Latent Action Enables Cross-Embodiment VLA Training from Human EgoVideos — Unlabeled human videos can effectively train VLA models by disentangling motion and action intent.
• GeneralVLA-2: Geometry-Aware Reconstruction and Governed Memory for Robot Planning — GeneralVLA-2 enhances robot planning through geometry-aware 3D reconstruction and a governed, quality-controlled long-term memory system.
• Pose6DAug: Physically Plausible Multi-view Object Swapping for Robot Data Augmentation — Pose6DAug enables VLA policy generalization by physically plausible 3D object swapping in successful robot episodes.
• VISTA: Vision-Grounded and Physics-Validated Adaptation of UMI data for VLA Training — Adapting UMI data for VLA models requires explicit visual grounding for fisheye views and physical validation for human trajectories.
• Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs — Decoupling physical motor skill learning from language grounding enables VLA models to scale with less expert data.
• VLAFlow: A Unified Training Framework for Vision-Language-Action Models via Co-training and Future Latent Alignment — Combining language supervision and future latent alignment stabilizes VLA model transfer performance across heterogeneous robot data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.