Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulation

2026-06-17 · Source: Artificial Intelligence · Field: Technology & Digital — Robotics & Autonomous Systems, Artificial Intelligence & Machine Learning · Depth: Advanced, quick

Summary

A study evaluated the impact of action space choices on motion smoothness, safety, and task performance in real-world reinforcement learning for vision-based robotic manipulation. Researchers benchmarked four action spaces: pose increment, pose velocity, joint position increment, and joint velocity. These were tested across two vision-based manipulation tasks, object picking and pushing, using policies trained in simulation and deployed to the real world via sim-to-real transfer. The findings indicate that action-space representation significantly influences sim-to-real performance. Specifically, the joint velocity action space demonstrated superior results for both picking and pushing tasks, excelling in terms of motion smoothness and overall final task performance. The study also offers practical guidance for RL practitioners.

Key takeaway

For Machine Learning Engineers developing vision-based robotic manipulation systems, you should prioritize the joint velocity action space. This choice significantly enhances sim-to-real transfer performance, leading to improved motion smoothness and overall task success in real-world object picking and pushing applications. Integrating joint velocity early in your simulation and deployment strategy can streamline development and boost operational reliability.

Key insights

Joint velocity action space significantly improves sim-to-real performance for vision-based robotic picking and pushing tasks.

Principles

• Action-space choice critically impacts sim-to-real performance.
• Joint velocity excels in smoothness and task performance.

Method

Policies were trained in simulation, then deployed to real-world robots using sim-to-real transfer for object picking and pushing tasks.

In practice

• Prioritize joint velocity for vision-based picking.
• Consider joint velocity for robotic pushing tasks.

Topics

• Reinforcement Learning
• Robotic Manipulation
• Action Spaces
• Sim-to-Real Transfer
• Vision-based Robotics
• Joint Velocity

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Robotics Engineer, Machine Learning Engineer

Related on AIssential

• Video Friday: Digit Learns to Dance—Virtually Overnight — Advanced robotics are achieving new performance thresholds in control, learning, and human interaction.
• ActionMap: Robot Policy Learning via Voxel Action Heatmap — Spatially structured voxel heatmaps for robot actions significantly improve VLA model performance and data efficiency.
• TempoVLA: Learning Speed-Controllable Vision-Language-Action Policies — Robot execution speed can be bidirectionally controlled by explicitly scaling predicted action magnitudes within Vision-Language-Action models.
• Table tennis robot defeats some of world’s best players – why this has major implications for robotics — Ace robot demonstrates AI's ability to achieve superhuman performance in complex, real-world physical environments.
• Japan Science and Technology Agency Develops NVIDIA-Powered Moonshot Robot for Elderly Care — Robots can predict 3D interaction forces from RGB images using simulation-trained AI.
• Making Foresight Actionable: Repurposing Representation Alignment in World Action Models — Aligning visual and action representations in World Action Models improves robot manipulation performance and robustness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.