HST-HGN: Heterogeneous Spatial-Temporal Hypergraph Networks with Bidirectional State Space Models for Global Fatigue Assessment

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

HST-HGN, a novel Heterogeneous Spatial-Temporal Hypergraph Network, is proposed for real-time driver fatigue assessment from untrimmed videos, addressing challenges in modeling long-range temporal dependencies and high-order facial synergies under constrained computational budgets. It integrates a hierarchical hypergraph network to fuse pose-disentangled geometric topologies with multi-modal texture patches, capturing high-order synergistic facial deformations. Temporally, a Bi-Mamba module with linear complexity, O(T), performs bidirectional sequence modeling, distinguishing ambiguous transient actions like yawning versus speaking. Evaluated on diverse fatigue benchmarks including YawDD, UTA-RLDD, FatigueView, and DMD, HST-HGN achieves state-of-the-art performance, with 98.57% accuracy and a Macro F1-Score of 98.28% on YawDD. It demonstrates high computational efficiency, requiring only 299 K trainable parameters and 2.90 G FLOPs per 128-frame clip, making it well-suited for in-cabin edge deployment.

Key takeaway

For Machine Learning Engineers developing in-cabin driver monitoring systems, HST-HGN offers a robust and computationally efficient solution for fatigue assessment. You should consider its heterogeneous hypergraph and Bi-Mamba architecture to accurately capture subtle facial cues and long-range temporal dependencies, even on resource-constrained edge devices. This approach enables precise differentiation of ambiguous actions like yawning from speaking, significantly improving detection reliability and real-time performance.

Key insights

HST-HGN uses hypergraphs and Bi-Mamba for efficient, accurate driver fatigue detection by modeling high-order spatial synergies and long-range temporal dependencies.

Principles

• High-order facial synergies improve fatigue detection.
• Linear complexity temporal models are crucial for edge devices.
• Pose-invariant features enhance robustness.

Method

HST-HGN employs global sparse sampling, 3D canonical alignment for geometry, Micro-CNN for texture, a dynamic hypergraph for multimodal fusion, and a Bi-Mamba block for bidirectional temporal evolution modeling.

In practice

• Deploy real-time fatigue detection on edge devices.
• Distinguish subtle actions like yawning versus speaking.

Topics

• Driver Fatigue Detection
• Hypergraph Neural Networks
• State Space Models
• Bi-Mamba
• Edge AI Deployment
• Facial Expression Analysis

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• HYolo: An Intelligent IoT-Based Object Detection System Using Hypergraph Learning — HYolo integrates hypergraph learning into YOLO to model complex high-order relationships, improving object detection in IoT.
• AHAN: Asymmetric Hierarchical Attention Network for Identical Twin Face Verification — AHAN improves identical twin face verification by analyzing multi-granularity and asymmetric facial features.
• HDST-GNN: Heterogeneous Dynamic Spatiotemporal Graph Neural Networks for Multi-Object Tracking in UAV Aerial Imagery — HDST-GNN improves multi-object tracking in UAV imagery by modeling heterogeneous object states and dynamic spatial context.
• Physics-Guided Attention in a Lightweight TCN for Efficient WiFi CSI-Based Human Activity Recognition — Incorporating physics-guided inductive biases into lightweight TCNs significantly improves WiFi CSI-based HAR efficiency and performance.
• H2VLR: Heterogeneous Hypergraph Vision-Language Reasoning for Few-Shot Anomaly Detection — H2VLR uses a heterogeneous hypergraph for high-order visual-semantic reasoning in few-shot anomaly detection.
• I\textsuperscript{2}RiMA: Spectral Riemannian Representation with Temporal Attention for Mental Stress Detection based on EEG Signals — I²RiMA uses spectral Riemannian representation and temporal attention for robust, efficient EEG-based mental stress detection.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.