Vision-Based Human Awareness Estimation for Enhanced Safety and Efficiency of AMRs in Industrial Warehouses

2025-06-10 · Source: cs.CV updates on arXiv.org · Field: Manufacturing & Industrial — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Smart Manufacturing & Industry 4.0 · Depth: Advanced, long

Summary

A new vision-based method estimates human awareness of autonomous mobile robots (AMRs) in industrial warehouses using a single RGB camera. This real-time pipeline, validated in NVIDIA Isaac Sim, integrates YOLO person detection, RTMW 3D pose lifting, and head orientation estimation to determine a human's 3D position and viewing cone relative to the AMR. The system then calculates a continuous "awareness score" (alpha) from 0 to 1, indicating if the human is looking at the robot. This allows AMRs to dynamically adjust their navigation, such as slowing down only when a human is unaware, thereby enhancing both safety and operational efficiency in mixed human-robot environments. The pipeline runs at 20 FPS on a consumer GPU.

Key takeaway

For Computer Vision Engineers developing AMR navigation systems, incorporating human awareness estimation can significantly improve both safety and efficiency. Your AMRs can transition from treating humans as generic obstacles to dynamically adjusting behavior based on real-time awareness, enabling more assertive and efficient paths when humans are attentive, and safer, slower movements when they are not. Consider implementing a vision pipeline that leverages 3D pose and head orientation to generate a continuous awareness score.

Key insights

Estimating human awareness via vision allows AMRs to adapt behavior, improving safety and efficiency in shared spaces.

Principles

• Head orientation indicates human attention to AMRs.
• Awareness enables dynamic AMR navigation adjustments.

Method

The pipeline uses YOLO for person detection, RTM3D for 2D/3D keypoint extraction, PnP for head pose recovery, and a geometry-based classifier to calculate an "awareness score" based on the AMR's position within the human's attention cone.

In practice

• Integrate awareness scores into AMR navigation stacks.
• Use NVIDIA Isaac Sim for robotics simulation and validation.

Topics

• Vision-Based Awareness Estimation
• Autonomous Mobile Robots
• Industrial Warehouse Automation
• 3D Human Pose Lifting
• Head Orientation Analysis

Code references

• open-mmlab/mmpose

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• Multi-HMR 2: Multi-Person Camera-Centric Human Detection, Mesh Recovery and Tracking — Multi-HMR 2 integrates camera-centric detection, mesh recovery, and tracking for robust 3D human perception.
• Computer Vision-Based Worker Safety Compliance — Computer vision provides continuous, AI-driven monitoring to enhance worker safety and compliance.
• A Machine Learning Framework for Real-Time Personalized Ergonomic Pose Analysis — Real-time personalized ergonomic pose analysis leverages 3D volumetric data to overcome 2D camera limitations and improve workplace safety.
• Stereo Vision-Based Fall Prediction and Detection using Human Pose Estimation on the AMD Kria K26 SOM — Privacy-preserving, real-time fall detection is feasible on edge devices using stereo vision and quantized human pose estimation.
• Creating Instant Inventories with AI: How a Simple Walkthrough Can Replace Manual Item Tracking — AI-powered tools automate inventory creation and updates, replacing manual data entry with visual and voice recognition.
• The Best Manufacturers Build AI with Workers, Not for Them — Successful AI integration in manufacturing requires co-creation with workers, contextual training, and performance-based measurement.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.