Ultrasound imaging turns a robot hand into a skillful mimic

2026-06-23 · Source: MIT Technology Review · Field: Technology & Digital — Robotics & Autonomous Systems, Artificial Intelligence & Machine Learning · Depth: Fundamental Awareness, quick

Summary

MIT researchers, led by mechanical engineering professor Xuanhe Zhao, have developed a novel wristband equipped with a miniaturized ultrasound "sticker" and hydrogel to track human hand dexterity. This device captures ultrasound images of the wrist's muscles, tendons, and ligaments as the wearer moves their hand. An artificial-intelligence algorithm, trained on meticulously labeled ultrasound images, continuously translates these images into the corresponding positions of the five fingers and palm in real time. Demonstrations show the wristband wirelessly controlling a robotic hand to mimic gestures, play a simple tune on a piano, shoot a mini basketball, and manipulate virtual objects on a computer screen, such as pinching to enlarge or minimize. The team plans to miniaturize the hardware and expand the AI training dataset to include diverse hand movements.

Key takeaway

For AI Engineers developing dexterous robotic systems or VR/AR interfaces, this wearable ultrasound technology offers a direct, high-fidelity input method. You can move beyond traditional motion capture by integrating real-time muscle and tendon imaging, enabling more nuanced and intuitive control. Consider exploring this approach for applications requiring fine motor skills, such as surgical robotics or advanced virtual object manipulation, to enhance precision and user experience.

Key insights

Wearable ultrasound imaging of wrist tendons enables real-time, high-dexterity control of robotic and virtual hands.

Principles

• Muscle and tendon states predict finger and palm positions.
• AI algorithms can translate biological signals into robotic actions.
• Wearable imaging offers intuitive control for dexterous tasks.

Method

A wristband with an ultrasound sticker images wrist muscles/tendons; an AI algorithm, trained on labeled images, translates these into real-time finger/palm positions.

In practice

• Wirelessly control robotic hands for delicate tasks.
• Manipulate virtual objects in VR/design applications.
• Train humanoid robots for surgical procedures.

Topics

• Robotics
• Human-Robot Interaction
• Ultrasound Imaging
• Wearable Technology
• Artificial Intelligence
• Dexterous Manipulation

Best for: Research Scientist, Robotics Engineer, AI Scientist, AI Engineer

Related on AIssential

• Wristband enables wearers to control a robotic hand with their own movements — An ultrasound wristband combined with AI precisely tracks hand movements for robotic control and virtual interaction.
• Can a Prosthetic Hand Deliver Humanoid Dexterity? — Integrating human-proven AI bionic hands into humanoids enhances factory-ready durability and dexterity.
• Dense Force Estimation with an Event-based Optical Tactile Sensor — Event-based tactile sensors can reconstruct dense 3D force fields by mapping displacement data to forces via iFEM and neural networks.
• Do as I Do: Dexterous Manipulation Data from Everyday Human Videos — DO AS I DO reconstructs and retargets human video interactions to generate scalable, executable data for dexterous robotic manipulation.
• AI-powered robot beats elite table tennis players — Sony AI's Ace robot demonstrates advanced real-time perception and control in complex human-robot interaction.
• Periodic Steady-State Control of a Handkerchief-Spinning Task Using a Parallel Anti-Parallelogram Tendon-driven Wrist — A novel tendon-driven wrist and hierarchical control enable precise, periodic manipulation of flexible objects like spinning handkerchiefs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by MIT Technology Review.