Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories

2026-07-23 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

Xiaomi-Robotics-1 is a foundational vision-language-action (VLA) model designed for mobile manipulation tasks in unseen environments and efficient adaptation to new tasks. It employs a two-stage training process, beginning with pre-training on over 100,000 hours of real-world manipulation trajectories collected via UMI devices, utilizing a scalable auto-labeling pipeline for scene state transitions. Post-training then aligns these capabilities with robot embodiments and imperative human instructions, using approximately 10,000 hours of cross-embodiment data. The model, built on a Mixture-of-Transformers architecture with Qwen3-VL and a diffusion transformer, demonstrates strong scaling behavior with increased data and model sizes (2B, 5B, 10B variants). It achieves a 57.6% success rate on RoboCasa365, surpassing the previous best of 46.6%, and an average score of 20.07 on RoboDojo, significantly outperforming the prior 13.07. Furthermore, it adapts to novel tasks with a 75% average success rate using less than 10 hours of data per task.

Key takeaway

For robotics engineers or AI scientists developing generalist robot policies, you should prioritize large-scale, diverse real-world data collection, especially using UMI devices. Implement a two-stage training approach, focusing on auto-labeling pipelines to scale data annotation and leveraging cross-embodiment post-training for aligning models to imperative instructions. This strategy significantly boosts your model's out-of-the-box performance in unseen environments and improves data efficiency for adapting to novel tasks.

Key insights

Scaling real-world manipulation data and model size significantly improves VLA robot performance and generalization.

Principles

• Data volume is a primary bottleneck for VLA generalization.
• Stronger pre-trained models yield better out-of-the-box performance.
• Auto-labeling pipelines enable large-scale trajectory annotation.

Method

Two-stage training: pre-train on 100k+ hours of UMI data with auto-labeled state transitions, then post-train on 10k+ hours of cross-embodiment data to align with robot actions and imperative instructions.

In practice

• Use UMI devices for scalable, diverse real-world data collection.
• Employ VLMs (e.g., Qwen3.5-27B) for auto-labeling trajectory segments.
• Mask missing action dimensions for heterogeneous robot embodiments.

Topics

• Vision-Language-Action Models
• Robot Foundation Models
• Mobile Manipulation
• Data Scaling
• Auto-labeling Pipelines
• UMI Devices
• Cross-embodiment Transfer

Code references

• Physical-Intelligence/openpi

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• VLAFlow: A Unified Training Framework for Vision-Language-Action Models via Co-training and Future Latent Alignment — Combining language supervision and future latent alignment stabilizes VLA model transfer performance across heterogeneous robot data.
• The Moving Eye: Enhancing VLA Spatial Generalization via Hybrid Dynamic Data Collection — Hybrid camera motion and diverse static viewpoints enhance VLA spatial generalization by reducing shortcut learning.
• What matters in building vision–language–action models for generalist robots — RoboVLMs provide a flexible, high-performing framework for generalist robot control by optimizing VLM integration.
• VISTA: Vision-Grounded and Physics-Validated Adaptation of UMI data for VLA Training — Adapting UMI data for VLA models requires explicit visual grounding for fisheye views and physical validation for human trajectories.
• TempoVLA: Learning Speed-Controllable Vision-Language-Action Policies — Robot execution speed can be bidirectionally controlled by explicitly scaling predicted action magnitudes within Vision-Language-Action models.
• Let It Be Simple: One-Step Action Generation for Vision-Language-Action Models — VLA action generation's unique structure allows effective one-step prediction by biasing diffusion training towards high-noise states.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.