Fusion-E2Pulse: A Multimodal Event-RGB Fusion Network for Non-contact Pulse Wave Reconstruction

2026-06-14 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

Fusion-E2Pulse is a novel multimodal network designed for non-contact pulse wave reconstruction, specifically addressing the challenge of accurately recovering waveform morphology, including the dicrotic notch. Traditional RGB-based methods, which extract physiological signals from facial videos, suffer from a smoothing effect during exposure that attenuates subtle vascular pulsation details. Conversely, neuromorphic event cameras, while highly sensitive to intensity fluctuations, are prone to noise and artifacts from minor motion. Fusion-E2Pulse combines frame-based integration and event-based differential sensing by using filtered RGB signals as structural priors to suppress motion artifacts, and utilizing event streams' high sensitivity to recover fine-grained morphological details. This framework achieves superior performance, effectively balancing noise suppression and morphological fidelity, with a mean absolute error of 0.78 bpm for heart rate estimation, a waveform correlation of 0.89, and a systolic phase duration error of 16.74 ms, demonstrating its efficacy in reconstructing fine-grained pathological features.

Key takeaway

For Computer Vision Engineers developing non-contact physiological monitoring systems, Fusion-E2Pulse demonstrates a superior approach to overcome the inherent limitations of single-modality sensing. You should consider integrating multimodal event-RGB fusion to enhance the accuracy of pulse wave reconstruction, particularly for critical features like the dicrotic notch. This method improves heart rate estimation to 0.78 bpm MAE and waveform correlation to 0.89, offering more reliable data for diagnostic applications.

Key insights

Fusing RGB and event camera data overcomes individual limitations for accurate non-contact pulse wave reconstruction.

Principles

• RGB signals provide structural priors.
• Event streams capture fine-grained details.
• Multimodal fusion balances noise and fidelity.

Method

Fusion-E2Pulse uses filtered RGB signals to suppress motion artifacts and event streams to recover fine morphological details, combining frame-based integration with event-based differential sensing.

In practice

• Improve heart rate estimation accuracy.
• Enhance pulse waveform correlation.
• Reduce systolic phase duration error.

Topics

• Multimodal Fusion
• Event Cameras
• RGB Imaging
• Pulse Wave Reconstruction
• Non-contact Sensing
• Physiological Monitoring

Best for: Research Scientist, AI Scientist, Computer Vision Engineer, Machine Learning Engineer

Related on AIssential

• Illumination-Robust Camera-Based Heart-Rate Estimation for Physiological Sensing in Robots — Robust remote photoplethysmography (rPPG) for robots is achievable despite illumination variations.
• M3D-Net: Multi-Modal 3D Facial Feature Reconstruction Network for Deepfake Detection — M3D-Net enhances deepfake detection by fusing RGB and 3D-reconstructed facial features.
• NERVE: A Neuromorphic Vision and Radar Ensemble for Multi-Sensor Fusion Research — NERVE is a multi-sensor dataset bridging neuromorphic vision and radar for robust perception research.
• EvTexture++: Event-Driven Texture Enhancement for Video Super-Resolution — Event signals offer high-frequency details crucial for texture restoration and temporal consistency in video super-resolution.
• EgoPressDiff: Multimodal Video Diffusion for Egocentric UV-Domain Hand-Pressure Estimation — EgoPressDiff uses multimodal video diffusion and specialized feature fusion to accurately estimate egocentric hand-pressure maps.
• Event-Adaptive State Transition and Gated Fusion for RGB-Event Object Tracking — MambaTrack dynamically adapts state transitions and fuses RGB-event data based on event density for robust object tracking.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.