What matters in building vision–language–action models for generalist robots

2026-02-11 · Source: Nature Machine Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, long

Summary

A new family of vision-language-action models (VLAs), named RoboVLMs, has been developed to enhance robotic manipulation by integrating foundation vision-language models (VLMs) with action components. This research, published in Nature Machine Intelligence in February 2026, identifies key factors influencing VLA performance, including backbone selection, VLA architecture formulation, and the timing of cross-embodiment data integration. Through extensive experiments involving over 8 VLM backbones, 4 policy architectures, and more than 600 distinct experiments, RoboVLMs achieved new state-of-the-art performance in three simulation tasks and real-world scenarios with minimal manual design. The highly flexible RoboVLMs framework, supporting easy integration of new VLMs and various design choices, is open-sourced, including codes, models, datasets (CALVIN, OXE, BDRBench20), and detailed training/evaluation recipes at robovlms.github.io.

Key takeaway

For AI Scientists and Robotics Engineers developing generalist robots, this research provides a comprehensive guide and an open-source framework, RoboVLMs, that simplifies VLA design and achieves superior performance. You should consider adopting RoboVLMs to streamline your VLA development, leveraging its flexibility for integrating new VLMs and optimizing architecture choices, especially when aiming for robust real-world robot manipulation.

Key insights

RoboVLMs provide a flexible, high-performing framework for generalist robot control by optimizing VLM integration.

Principles

• Cross-embodiment data improves few-shot learning.
• Mix-of-Expert (MoE) structures enhance VLA generalization.
• VLA design choices significantly impact robot manipulation performance.

Method

The RoboVLMs framework systematically evaluates VLM backbones, policy architectures, and cross-embodiment data integration to achieve optimal VLA performance in robotic manipulation tasks.

In practice

• Utilize RoboVLMs for generalist robot control.
• Integrate cross-embodiment data for better few-shot learning.
• Explore MoE architectures for VLA generalization.

Topics

• Vision-Language-Action Models
• Generalist Robotics
• RoboVLMs
• Cross-Embodiment Learning
• Robot Manipulation

Code references

• mees/calvin
• google-deepmind/open_x_embodiment
• Robot-VLAs/RoboVLMs
• vikhyat/moondream

Best for: Computer Vision Engineer, AI Scientist, Research Scientist, AI Researcher, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• G$^3$VLA: Geometric inductive bias for Vision-Language-Action Models — G$^3$VLA injects calibrated 3D geometry into VLA models' visual streams, improving robot manipulation by resolving 2D-3D coordinate mismatch.
• True Positive Weekly #142 — Robotic foundation models leverage multimodal LLMs and specialized control modules for dexterous, generalizable robotic task execution.
• A better method for planning complex visual tasks — A dual-VLM framework combines visual understanding with formal planning for robust, generalizable long-term task execution.
• LabVLA: Grounding Vision-Language-Action Models in Scientific Laboratories — Scientific lab automation requires VLA models grounded in specialized data and diverse robot embodiments, addressed by simulation-driven data generation and two-stage policy training.
• ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning — Unified data, action manifold learning, and modular 3D perception enable generalizable robotic manipulation across diverse hardware.
• VLANeXt: The Design Recipes Behind Vision-Language-Action Robots — VLANeXt provides design recipes for integrating vision, language, and action in robots.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Nature Machine Intelligence.