Beyond English: Uncovering the Multilingual Gap in Vision-Language-Action Models

2026-06-14 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

A systematic study reveals a significant "multilingual gap" in Vision-Language-Action (VLA) models, which learn robot policies from multimodal data. VLA systems trained primarily on English instructions show substantial performance degradation when evaluated with other languages. This occurs even though underlying large language models often possess multilingual capabilities. Researchers constructed multilingual instructions by translating existing benchmarks and assessed several VLA models in simulation settings. Findings indicate performance drops correlate with both instruction understanding and action execution. Representation shifts also contribute to this gap. To address this, a novel "Multilingual Principal Component Alignment" (MPCA) approach is proposed. This method leverages Principal Component Analysis to align projected multilingual representations, effectively reducing the performance disparity.

Key takeaway

For Robotics Engineers developing Vision-Language-Action (VLA) models for global deployment, recognize that multilingual capabilities do not automatically transfer from underlying LLMs. Your VLA models will likely suffer significant performance degradation on non-English instructions. You should explicitly address this multilingual gap. Construct diverse language datasets and consider methods like Multilingual Principal Component Alignment (MPCA) to align representations. This ensures robust performance across languages.

Key insights

Vision-Language-Action models exhibit a significant multilingual performance gap, which can be mitigated by aligning multilingual representations.

Principles

• VLA performance drops correlate with instruction understanding.
• Action execution also impacts VLA multilingual performance.
• Representation shifts cause the multilingual gap.

Method

Multilingual Principal Component Alignment (MPCA) uses Principal Component Analysis to identify a principal component subspace. It then aligns projected multilingual representations to reduce performance gaps.

In practice

• Translate benchmarks for multilingual VLA evaluation.
• Evaluate VLA models on diverse language instructions.
• Apply MPCA to align multilingual representations.

Topics

• Vision-Language-Action Models
• Multilingual AI
• Robot Learning
• Principal Component Analysis
• Cross-lingual Transfer
• Representation Alignment

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• Continuous Reasoning for Vision-Language-Action — Continuous reasoning, shareable and verifiable, bridges the granularity gap between language and continuous robot control.
• LARA: Latent Action Representation Alignment for Vision-Language-Action Models — LARA jointly optimizes Latent Action Models and Vision-Language-Action models via representation alignment for improved robot control.
• Let It Be Simple: One-Step Action Generation for Vision-Language-Action Models — VLA action generation's unique structure allows effective one-step prediction by biasing diffusion training towards high-noise states.
• When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs — VLAs often fail to follow language due to visual shortcuts; a new method mitigates this by comparing conditioned and unconditioned actions.
• Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Time — Adapting Vision-Language-Action models to new tasks can be achieved via demonstration retrieval, not costly per-task fine-tuning.
• Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models — DeepVision-VLA enhances robotic manipulation by integrating multi-level visual features and pruning irrelevant visual tokens.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.