Integrated Sensing and Communications for Real-time Avatar Control in XR over 5G

2026-06-22 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Emerging Technologies & Innovation · Depth: Advanced, quick

Summary

An integrated sensing and communication (ISAC) architecture is proposed for real-time avatar control in Extended Reality (XR) over 5G and 6G networks, addressing the limitations of current handheld controllers and cameras. This multimodal system combines 5G MillimeterWave (mmWave) ISAC with surface electromyography (sEMG) signals. 5G mmWave ISAC utilizes existing communication signals, specifically power-per-beam-pair (PPBP) from standard beam management, to derive coarse body-level gestures, achieving 82.2±5.9% average accuracy on unseen users. For fine-grained finger-level interactions, lightweight sEMG sensors capture forearm muscle activity. This combination enables multi-scale gesture recognition, forming a complete XR framework that supports seamless physical action translation to the virtual world.

Key takeaway

For AI Engineers and XR developers designing immersive experiences, this research presents a compelling approach to achieve natural, controller-free avatar control. You should consider integrating 5G mmWave ISAC with sEMG for robust, multi-scale gesture recognition. This method moves beyond traditional HMD and controller limitations, enhancing user immersion and interaction fidelity in future XR applications by translating physical actions seamlessly into the virtual world.

Key insights

Combining 5G mmWave ISAC and sEMG enables multi-scale gesture recognition for real-time XR avatar control, overcoming current limitations.

Principles

• 5G mmWave signals can derive coarse body poses.
• sEMG captures fine-grained finger gestures.
• Multimodal sensing improves XR interaction.

Method

The architecture uses 5G mmWave ISAC for body-level gestures via power-per-beam-pair (PPBP) from beam management, and sEMG sensors for finger-level gestures by capturing forearm muscle activity.

In practice

• Use 5G mmWave for body pose estimation.
• Deploy sEMG for detailed finger tracking.
• Integrate both for full XR avatar control.

Topics

• Integrated Sensing and Communications
• Extended Reality
• 5G mmWave
• Surface Electromyography
• Avatar Control
• Gesture Recognition

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• A Camera-Cooperative ISAC Framework for Multimodal Non-Cooperative UAVs Sensing — Multimodal sensing, combining camera and ISAC, significantly optimizes resource allocation for non-cooperative UAV detection and tracking.
• X-Band UAV-enabled Integrated Sensing and Communications for Vehicular Networks — Optimal time allocation in X-band UAV-enabled ISaC systems balances sensing accuracy and communication performance for vehicular networks.
• Spatial-Temporal Decoupled Adapter for Micro-gesture Online Recognition — Decoupling spatial and temporal adaptation with adaptive augmentation significantly improves micro-gesture recognition in imbalanced datasets.
• XEmbodied: A Foundation Model with Enhanced Geometric and Physical Cues for Large-Scale Embodied Environments — XEmbodied enhances VLA models with intrinsic 3D geometric and physical awareness for robust performance in complex embodied environments.
• Dense Force Estimation with an Event-based Optical Tactile Sensor — Event-based tactile sensors can reconstruct dense 3D force fields by mapping displacement data to forces via iFEM and neural networks.
• AVI-HT: Adaptive Vision-IMU Fusion for 3D Hand Tracking — Adaptive fusion of egocentric vision and on-glove 6-DoF IMUs, guided by cross-sensor attention, significantly improves 3D hand tracking under occlusion.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.