Illumination-Robust Camera-Based Heart-Rate Estimation for Physiological Sensing in Robots

2026-06-10 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

A new end-to-end spatial-temporal transformer framework has been developed for remote heart-rate (HR) estimation, specifically designed for physiological sensing in robots operating under varied illumination. This estimator integrates PRNet-based 3D face alignment, clip-level illumination augmentation, and a Residual Temporal Standardization Module. Its training objective combines a Soft-Shifted Pearson waveform loss with a spectral Kullback-Leibler divergence loss, using a tuned weight (β) for frequency-domain guidance. Experiments on a static all-level mix protocol across three illumination levels showed that β=5 yielded the strongest results, achieving an HR mean absolute error (MAE) of 0.79 bpm and an HR correlation of 0.982. This represents a 93.6% reduction in HR MAE and an increase in HR correlation from 0.088 to 0.982 compared to the PhysFormer baseline on the same dataset.

Key takeaway

For Robotics Engineers developing service or assistive robots requiring accurate physiological sensing, this illumination-robust framework offers a significant advancement. If your robot-mounted vision systems struggle with non-contact heart-rate estimation in diverse lighting, consider integrating this transformer-based approach. Its demonstrated performance, with a 0.79 bpm MAE and 0.982 correlation, suggests a reliable solution for real-world human-robot interaction scenarios.

Key insights

Robust remote photoplethysmography (rPPG) for robots is achievable despite illumination variations.

Principles

• Illumination variation is a major barrier for rPPG deployment.
• Hybrid temporal-frequency supervision improves HR estimation.
• Combined loss functions enhance robustness in varied conditions.

Method

An end-to-end spatial-temporal transformer framework integrates PRNet-based 3D face alignment, clip-level illumination augmentation, and a Residual Temporal Standardization Module, trained with a Soft-Shifted Pearson waveform loss and spectral Kullback-Leibler divergence loss.

In practice

• Use PRNet for 3D face alignment in rPPG.
• Apply clip-level illumination augmentation.
• Combine waveform and spectral losses for HR estimation.

Topics

• Remote Photoplethysmography
• Heart-Rate Estimation
• Robotics
• Computer Vision
• Illumination Robustness
• Transformers

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• FS-DVS: A Frequency-Selective Dynamic Visual Sensing Paradigm for Enhancing Information Completeness — FS-DVS uses a learnable pre-trigger spatial filter to mimic RGCs, enhancing event camera data completeness and noise resilience.
• CHOIR: Contact-aware 4D Hand-Object Interaction Reconstruction — CHOIR reconstructs 4D hand-object interactions from monocular video, explicitly using contact as a coupling signal for improved consistency.
• Privacy-Preserving Clothing Classification using Vision Transformer for Thermal Comfort Estimation — ViT-based encryption maintains high accuracy for privacy-preserving image classification, unlike pixel-based methods.
• RPCASSM: Robust PCA State Space Model For Infrared Small Target Detection — RPCASSM leverages Robust PCA to design specialized state space modules for background and target, accurately detecting infrared small targets by their spatial properties.
• Ambient-robust Inverse Rendering using Active RGB-NIR Imaging — Combining active NIR flash with multi-view RGB imaging yields ambient-robust inverse rendering for accurate geometry and reflectance.
• Physics-Informed Conditional Diffusion for Motion-Robust Retinal Temporal Laser Speckle Contrast Imaging — RetinaDiff uses physics-informed diffusion to reconstruct robust retinal blood flow from minimal speckle frames.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.