Multi-HMR 2: Multi-Person Camera-Centric Human Detection, Mesh Recovery and Tracking

2026-06-12 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Computer Vision · Depth: Expert, quick

Summary

Multi-HMR 2, published on 2026-06-12, is a robust DETR-based framework designed for multi-person camera-centric human detection, mesh recovery, and tracking. It addresses limitations of traditional pelvis-centered approaches by focusing on metric 3D localization and detection accuracy in the camera coordinate system. The system predicts a scene-consistent camera alongside human meshes, enabling metric 3D localization without requiring ground-truth intrinsics. Furthermore, by distilling image-based memory features from SAM2, Multi-HMR 2 extends to tracking, achieving consistent identity association without video supervision. Despite its conceptual simplicity, lacking handcrafted components, video input, or ground-truth cameras, it achieves state-of-the-art pelvis-centered performance while substantially improving detection accuracy and metric 3D localization.

Key takeaway

For Computer Vision Engineers developing multi-person 3D perception systems, Multi-HMR 2 offers a robust, simplified approach. You can achieve accurate metric 3D localization and consistent identity tracking without relying on complex video supervision or handcrafted components. Consider evaluating this framework to streamline deployment and improve performance in dynamic, real-world environments, especially for human-robot interaction.

Key insights

Multi-HMR 2 integrates camera-centric detection, mesh recovery, and tracking for robust 3D human perception.

Principles

• Camera-centric recovery is key for real-world HMR applications.
• Image-based memory features enable consistent identity tracking without video supervision.

Method

Multi-HMR 2 is a DETR-based framework that predicts scene-consistent cameras and human meshes, using SAM2-distilled memory features for identity tracking.

In practice

• Develop human-robot interaction systems.
• Enhance social scene understanding applications.

Topics

• Human Mesh Recovery
• Multi-person Tracking
• 3D Human Localization
• Camera-centric Perception
• DETR
• SAM2

Best for: Research Scientist, AI Scientist, Computer Vision Engineer, Robotics Engineer

Related on AIssential

• Does Appearance Help? A Systematic Study of Image-Based Re-Identification in Online 3D Multi-Pedestrian Tracking — Integrating lightweight image-based ReID with LiDAR tracking via cascaded matching improves identity preservation in crowded 3D environments for mobile robots.
• Vision-Based Human Awareness Estimation for Enhanced Safety and Efficiency of AMRs in Industrial Warehouses — Estimating human awareness via vision allows AMRs to adapt behavior, improving safety and efficiency in shared spaces.
• TROPHIES: Temporal Reconstruction of Places, Humans, and Cameras from Multi-view Videos — TROPHIES unifies human, scene, and camera reconstruction from multi-view videos into a globally consistent 4D space.
• HumanSplatHMR: Closing the Loop Between Human Mesh Recovery and Gaussian Splatting Avatar — Jointly optimizing human pose and avatar rendering improves 3D geometry and generalization for in-the-wild video.
• SToRe3D: Sparse Token Relevance in ViTs for Efficient Multi-View 3D Object Detection — Jointly sparsifying 2D image tokens and 3D object queries with relevance-aligned filtering significantly boosts multi-view 3D detection efficiency.
• EgoPoseFormer v2: Accurate Egocentric Human Motion Estimation for AR/VR — EgoPoseFormer v2 combines a transformer model with an auto-labeling system for accurate, temporally consistent egocentric human motion estimation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.