Pelican-VLA 0.5: Attending Before Acting Benefits Generalization

2026-07-09 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

Pelican-VLA 0.5 is a unified Vision-Language-Action (VLA) model built on a Qwen3-VL 4B Transformer backbone, integrating vision-language understanding, future-frame generation, and action prediction. It introduces K=32 learnable "Reasoning Slots" between perception and action, which act as a bottleneck to induce manipulation-centric attention during pre-training. This enables "attention-level generalization," where the model's action pathway focuses on instruction-relevant objects and contact regions without explicit supervision or task-specific fine-tuning, even across unseen scenes and robot embodiments. Pre-trained on 2,400 hours of heterogeneous robot data, Pelican-VLA 0.5 achieves 91.4% success on RoboTwin Clean and 91.0% on RoboTwin Randomized after fine-tuning, outperforming other open-source VLA baselines. The model demonstrates early zero-shot manipulation capabilities, though a "representation-to-action gap" remains for precise control.

Key takeaway

For machine learning engineers developing generalizable robotic policies, consider integrating bottleneck architectures like Pelican-VLA 0.5's "Reasoning Slots." This approach can foster robust, manipulation-centric attention during pre-training, reducing reliance on extensive task-specific fine-tuning for object grounding. Focus subsequent fine-tuning on refining the "representation-to-action mapping" rather than re-learning visual attention, potentially accelerating deployment across diverse embodiments and tasks.

Key insights

Routing perception through reasoning slots induces manipulation-centric attention for VLA models.

Principles

• Constrained perception-action interfaces improve generalization.
• Attention-level generalization precedes action-level generalization.
• Fine-tuning refines existing representations, not creates them.

Method

Pelican-VLA 0.5 uses a Qwen3-VL 4B backbone with K=32 Reasoning Slots, a curriculum bottleneck mask, orthogonality regularizer, and slot-gated generation, trained with flow-matching action, future-frame, and slot-language contrastive losses.

In practice

• Implement reasoning slots as a perception-action bottleneck.
• Use curriculum learning for attention mask constraints.
• Combine flow-matching with contrastive and generation losses.

Topics

• Vision-Language-Action
• Robotics Generalization
• Reasoning Slots
• Attention Mechanisms
• RoboTwin Benchmark
• Embodied AI

Code references

• Open-X-Humanoid/Pelican-VLA05

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• Pelican-VLA 0.5: Attending Before Acting Benefits Generalization — Pelican-VLA 0.5 achieves robust attention-level generalization in robotics via learnable Reasoning Slots.
• See Less, Specify More: Visual Evidence Budgets for Generalizable VLAs — VLA generalization improves by training executors with explicit local guidance and task-sufficient visual evidence, avoiding weak supervision.
• Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models — Reinforcement learning post-training with GRPO significantly enhances VLA model performance on robotic manipulation tasks using public data.
• Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation — Integrating historical experience directly into a VLA model's native latent embedding space enhances long-horizon task performance.
• Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation — Instruction factor bias, where policies over-rely on dominant semantic components, hinders compositional generalization in robotic manipulation.
• What matters in building vision–language–action models for generalist robots — RoboVLMs provide a flexible, high-performing framework for generalist robot control by optimizing VLM integration.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.