SAM-Deep-EIoU: Selective Mask Propagation for Multi-Object Tracking

2026-06-11 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

SAM-Deep-EIoU is a novel multi-object tracking algorithm that addresses the challenge of "hard" frames where lightweight base trackers typically fail. It employs a selective mask propagation strategy, dispatching to a more computationally intensive video object segmentation (VOS) model only when an assignment-uncertainty signal indicates a potential tracking error. The system intelligently integrates VOS outputs: it modifies the base tracker's identity assignment exclusively when the VOS model provides a confident prediction that contradicts the base tracker. Conversely, weak or inconclusive VOS predictions do not alter the base output. This method is training-free, treats both the base tracker and VOS model as black boxes, and can readily incorporate more capable VOS components. It improved three distinct base trackers on DanceTrack and achieved state-of-the-art performance on SportsMOT with 86.8 HOTA using SAM3-Deep-EIoU with global track association.

Key takeaway

For Computer Vision Engineers optimizing multi-object tracking systems, SAM-Deep-EIoU provides a practical approach to enhance performance without extensive retraining. You should consider implementing selective mask propagation to improve the robustness of your base trackers, especially in scenarios with high assignment uncertainty. This allows you to selectively leverage powerful video object segmentation models for critical frames, boosting identity preservation and overall accuracy, as demonstrated by its state-of-the-art 86.8 HOTA on SportsMOT, while managing computational overhead.

Key insights

Selective mask propagation intelligently combines lightweight trackers with expensive VOS models to improve multi-object tracking efficiency and accuracy.

Principles

• Dispatch expensive models only on uncertainty signals.
• Confidently contradictory predictions override base outputs.
• Black-box integration allows component upgrades.

Method

The method dispatches a VOS model only when a base tracker's assignment-uncertainty signal fires, modifying the base output solely if the VOS provides a confident, contradictory prediction.

In practice

• Improve existing base trackers without retraining.
• Upgrade VOS component for better performance.
• Enhance identity preservation in sports analytics.

Topics

• Multi-Object Tracking
• Video Object Segmentation
• Selective Mask Propagation
• Computer Vision
• Sports Analytics
• HOTA Benchmark

Best for: Research Scientist, Computer Vision Engineer, AI Scientist

Related on AIssential

• SAM 3.1: Faster and More Accessible Real-Time Video Detection and Tracking With Multiplexing and Global Reasoning - AI at Meta — SAM 3.1 enhances real-time video object tracking via multiplexing, doubling processing speed and reducing GPU demands.
• Masked Diffusion Vision-Language Models for Temporal Action Localization — Masked diffusion vision-language models enable joint, iterative refinement of temporal boundaries and semantic content in untrimmed videos.
• HDST-GNN: Heterogeneous Dynamic Spatiotemporal Graph Neural Networks for Multi-Object Tracking in UAV Aerial Imagery — HDST-GNN improves multi-object tracking in UAV imagery by modeling heterogeneous object states and dynamic spatial context.
• Segment Anything with Motion, Geometry, and Semantic Adaptation for Complex Nonlinear Visual Object Tracking — SAMOSA adapts SAM 2 for robust visual object tracking by integrating explicit motion, geometry, and semantic cues.
• MoEIoU: Rethinking Bounding-Box Regression as a Mixture of Experts — MoEIoU adaptively prioritizes localization error components for improved bounding-box regression, leading to faster convergence and higher accuracy.
• A Probabilistic Framework for Improving Dense Object Detection in Underwater Image Data via Annealing-Based Data Augmentation — A pseudo-simulated annealing data augmentation method improves underwater object detection in crowded, natural scenes.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.