Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning

2026-06-10 · Source: Artificial Intelligence · Field: Technology & Digital — Robotics & Autonomous Systems, Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new framework for tactile-only blind grasping, deployable on physical multi-fingered robotic hands, addresses challenges like the sim-to-real gap and sparse tactile signals. The approach integrates three core components: a Real2Sim tactile calibration pipeline that generates a contact-calibrated digital-twin simulator, a layout-aware tactile encoder enhancing sparse tactile observations through self-supervised pretraining with sensor-geometry priors, and a method for training object-specific reinforcement learning experts in the calibrated simulator, aggregating their successful grasp trajectories into a tactile-conditioned Diffusion Policy for improved generalization. Evaluated on a physical LEAP Hand with distributed tactile sensing, the policy achieved a 27% real-world grasp success rate across 20 objects (10 seen, 10 unseen) without visual input or real-world demonstrations. Ablations confirmed the benefits of layout-aware pretraining and Real2Sim calibration.

Key takeaway

For robotics engineers developing dexterous manipulation systems, this research demonstrates a viable path to tactile-only blind grasping. You should consider integrating Real2Sim tactile calibration and geometry-aware tactile encoders to bridge the sim-to-real gap. Leveraging diffusion policies for aggregating simulated expert trajectories can significantly improve generalization for unseen objects, reducing reliance on visual input in complex environments.

Key insights

Combining calibrated simulation, geometry-aware tactile encoding, and diffusion policies enables blind dexterous grasping.

Principles

• Calibrating simulators to real tactile signals is crucial.
• Sensor geometry priors improve sparse tactile representation.
• Aggregating expert trajectories enhances policy generalization.

Method

The framework involves Real2Sim tactile calibration, self-supervised pretraining of a layout-aware tactile encoder, and training object-specific RL experts in simulation, then aggregating trajectories into a Diffusion Policy.

In practice

• Implement Real2Sim calibration for tactile sensor fidelity.
• Use layout-aware encoders for sparse tactile data.
• Employ Diffusion Policies for robust grasp trajectory aggregation.

Topics

• Blind Dexterous Grasping
• Tactile Sensing
• Real2Sim Calibration
• Diffusion Policy
• Robotic Manipulation
• LEAP Hand

Best for: Research Scientist, Robotics Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation — Center-of-Pressure (CoP) enables robust sim-to-real dexterous manipulation by preserving dense, physics-grounded tactile information.
• Bridging the Morphology Gap: Adapting VLA Models to Dexterous Manipulation via Intent-Conditioned Fine-Tuning — Adapting VLA models to dexterous hands is possible by decoupling control via a virtual grasp intent and a two-stage learning architecture.
• Dense Force Estimation with an Event-based Optical Tactile Sensor — Event-based tactile sensors can reconstruct dense 3D force fields by mapping displacement data to forces via iFEM and neural networks.
• TouchThinker: Scaling Tactile Commonsense Reasoning to the Open World with Large-scale Data and Action-aware Representation — TouchThinker scales open-world tactile commonsense reasoning through large-scale data and action-aware representation.
• Real2Sim: A Physics-driven and Editable Gaussian Splatting Framework for Autonomous Driving Scenes — Real2Sim combines 4D Gaussian Splatting with a differentiable MPM solver for physics-aware, editable autonomous driving scene generation.
• TAM: Torque Adaptation Module for Robust Motion Transfer in Manipulation — TAM adapts torque commands at the robot interface, enabling robust motion transfer and decoupling policy training from robot dynamics.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.