Bounding-Box Trajectories Matter for Video Anomaly Detection

2026-05-22 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, extended

Summary

TrajVAD is a novel framework for video anomaly detection (VAD) that primarily models multi-class bounding-box trajectories using normalizing flows to learn normal kinematic patterns. It addresses limitations of human pose-based methods by extending anomaly detection to non-human entities and maintaining robustness when pose estimation is unreliable. The trajectory-only variant, TrajVAD-T, achieved 87.7% AP on ShanghaiTech and the best results on MSAD, outperforming all compared pose-based methods while saving 31 ms per frame by eliminating pose estimation. An extended version, TrajVAD-P, integrates a reliability-gated pose branch, further boosting performance to 88.6% AUROC and 90.9% AP on ShanghaiTech. This framework demonstrates that bounding-box trajectories are an effective, yet underexplored, modality for VAD across diverse environments like ShanghaiTech, UBnormal, and MSAD.

Key takeaway

For Machine Learning Engineers developing video anomaly detection systems, particularly for multi-object or real-time applications, you should prioritize bounding-box trajectory analysis. This approach offers superior performance on non-human anomalies and maintains robustness under occlusion, outperforming pose-based methods. Consider implementing TrajVAD-T to achieve high accuracy with significant computational savings (31 ms/frame), or TrajVAD-P for enhanced human-centric anomaly detection when pose data is reliable. Your system's performance will be highly dependent on upstream detector quality.

Key insights

Bounding-box trajectories, a byproduct of detection and tracking, are a powerful, multi-class, and efficient primary signal for video anomaly detection.

Principles

• Bounding-box trajectories offer class-agnostic anomaly signals.
• Trajectory kinematics are robust to pose estimation failures.
• Detector confidence is a critical feature for trajectory-based VAD.

Method

TrajVAD uses normalizing flows to model multi-class bounding-box trajectory features, derived from smoothed coordinates, dimensions, and detector confidence, conditioned on class embeddings, to score anomalies via negative log-likelihood.

In practice

• Use multi-class bounding-box trajectories for VAD beyond human-centric scenes.
• Prioritize detector confidence as a key feature in trajectory-based models.
• Consider TrajVAD-T for real-time VAD due to its efficiency.

Topics

• Video Anomaly Detection
• Bounding Box Trajectories
• Normalizing Flows
• Multi-Class Tracking
• Real-time Inference
• Object Detection

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Bounding-Box Trajectories Matter for Video Anomaly Detection — Bounding-box trajectories offer an effective, underexplored modality for robust video anomaly detection, even without pose estimation.
• TraRA: Trajectory-level Recognition Aggregation for Video Text Spotting in Urban Surveillance — TraRA improves video text spotting by aggregating temporal and multimodal information at the trajectory level.
• DRIFT: From Robustness Gaps to Invariance Manifolds for AI-Generated Image Detection — AI-generated image detection can be framed as learning a real-image invariance manifold to identify margin violations.
• MoEIoU: Rethinking Bounding-Box Regression as a Mixture of Experts — MoEIoU adaptively prioritizes localization error components for improved bounding-box regression, leading to faster convergence and higher accuracy.
• Visual Prompting Meets Feature Reconstruction-Based Anomaly Detection with Dual-Teacher Supervision — Robust anomaly detection in varied real-world conditions is achieved by combining visual prompting, adaptive teacher models, and synthetic data.
• Online Goal Recognition using Path Signature and Dynamic Time Warping — Path signatures offer a compact, expressive way to encode trajectories for online goal recognition.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.