Human Universal Grasping

2026-06-15 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Data Science & Analytics · Depth: Expert, quick

Summary

Human Universal Grasping (HUG) introduces a flow-matching model designed to generate diverse human grasps for any user-specified object from a single RGB-D image. This model addresses the challenge of multi-fingered robots lacking human-level grasping generality. The authors collected 1M-HUGs, an egocentric dataset comprising 1 million frames (27.8 hours) and 6,707 object instances across 41 buildings. HUG fuses RGB and depth observations to output a grasp parameterized by wrist translation, rotation, and MANO hand pose. Predicted grasps can be retargeted for zero-shot robot grasping. Evaluated on HUG-Bench, a new simulated benchmark of 90 unseen objects, HUG outperforms state-of-the-art baselines by +23% and +34% on a challenging 30-object real-world test set.

Key takeaway

For robotics engineers developing multi-fingered grasping systems, HUG offers a robust, data-driven solution to overcome limitations in generality. Your teams can utilize the HUG model and its 1M-HUGs dataset to train robots for zero-shot grasping in complex, everyday environments. Consider integrating this approach to significantly improve robot manipulation capabilities, as it outperforms existing baselines by up to +34%.

Key insights

Human egocentric grasping data and flow-matching models enable robots to achieve universal, zero-shot object manipulation.

Principles

• Human egocentric data is a natural source for robot grasping.
• Fusing RGB-D observations improves grasp generation.
• Retargeting human grasps enables zero-shot robot manipulation.

Method

Collect 1M-HUGs egocentric human grasp data using smart glasses. Train a flow-matching model that fuses RGB and depth to output wrist translation, rotation, and MANO hand pose for grasp generation.

In practice

• Generate diverse human grasps for any object from RGB-D.
• Retarget predicted grasps to various robot hands.
• Evaluate robot grasping using the HUG-Bench benchmark.

Topics

• Robot Grasping
• Human Universal Grasping
• Flow-Matching Models
• Egocentric Data
• RGB-D Perception
• Zero-Shot Learning

Best for: Computer Vision Engineer, Research Scientist, Robotics Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• 🕷️Human Universal Grasping🕷️ 👉HUG is a flow-matching model that generates diverse... — HUG is a flow-matching model generating diverse human grasps from a single RGB-D image.
• HRDexDB: A Large-Scale Dataset of Dexterous Human and Robotic Hand Grasps — HRDexDB is a multi-modal dataset for dexterous grasping, combining human and robotic hand data.
• UMI-3D: Extending Universal Manipulation Interface from Vision-Limited to 3D Spatial Perception — Integrating LiDAR into a wrist-mounted manipulation interface significantly enhances data quality and task reliability for embodied robot learning.
• UniDexTok: A Unified Dexterous Hand Tokenizer from Real Data — UniDexTok unifies heterogeneous dexterous hand data into a shared discrete token space, enabling accurate cross-embodiment state representation and transfer.
• What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos? — Cotraining robot policies with everyday human videos requires high-quality hand pose data and specialized networks to bridge the motion gap.
• Bounding Boxes as Goals: Language-Conditioned Grasping via Neuro-Symbolic Planning — GRASP enables robots to interpret natural language and perform open-vocabulary grasping using VLMs and bounding boxes, achieving 73.3% success without fine-tuning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.