ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning

2026-07-02 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Computer Vision · Depth: Expert, quick

Summary

ICDepth is a novel framework designed to improve monocular video depth estimation by adapting pre-trained text-to-video diffusion transformers. It addresses the common challenges of achieving temporal consistency, geometric accuracy, and generalization simultaneously, which existing discriminative and generative methods struggle with. ICDepth leverages In-Context Conditioning (ICC) and introduces SAND-Attention for precise spatial-temporal alignment and SRFM to inject DINOv2 semantic and resolution priors. This approach enables ICDepth to achieve state-of-the-art results on multiple benchmarks with remarkable data efficiency, requiring only 0.8M frames for training, which is 6-13 times less than competing generative methods, while also demonstrating strong zero-shot generalization across diverse domains.

Key takeaway

For Computer Vision Engineers developing video depth estimation solutions, ICDepth offers a compelling approach to achieve state-of-the-art performance with significantly reduced data requirements. You should consider exploring diffusion transformer architectures and in-context conditioning to improve temporal consistency and generalization in your models, especially if data scarcity is a concern. This method allows for robust zero-shot generalization, potentially streamlining deployment across varied environments.

Key insights

ICDepth adapts video diffusion transformers for accurate, data-efficient video depth estimation using in-context conditioning.

Principles

• Leverage pre-trained diffusion models for dense prediction tasks.
• Ensure precise spatial-temporal alignment via shared RoPE and unidirectional attention.
• Inject semantic and resolution priors to enhance geometric precision.

Method

ICDepth adapts text-to-video diffusion transformers via In-Context Conditioning, using SAND-Attention for spatial-temporal alignment and SRFM to inject DINOv2 semantic and resolution priors.

In practice

• Apply diffusion models to dense prediction tasks like depth estimation.
• Integrate DINOv2 features to improve geometric accuracy.
• Reduce training data requirements for video-based models.

Topics

• Video Depth Estimation
• Diffusion Models
• In-Context Learning
• Computer Vision
• Temporal Consistency
• DINOv2

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Masked Diffusion Vision-Language Models for Temporal Action Localization — Masked diffusion vision-language models enable joint, iterative refinement of temporal boundaries and semantic content in untrimmed videos.
• 3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation — Recasting monocular video depth as multi-view 3D reconstruction with 3D consistency optimization improves geometric accuracy and coherence.
• EgoPressDiff: Multimodal Video Diffusion for Egocentric UV-Domain Hand-Pressure Estimation — EgoPressDiff uses multimodal video diffusion to generate continuous, temporally consistent UV-pressure maps from egocentric video.
• Beyond Skeletons: Learning Animation Directly from Driving Videos with Same2X Training Strategy — DirectAnimator bypasses pose estimation in HIA by using raw video cues and a specialized training strategy for superior results.
• Diff-SBSR: Learning Multimodal Feature-Enhanced Diffusion Models for Zero-Shot Sketch-Based 3D Shape Retrieval — Diffusion models, enhanced with multimodal features, can effectively perform zero-shot sketch-based 3D shape retrieval.
• Paper Digest: ICCV 2025 Papers & Highlights — Multimodal AI, 3D reconstruction, and video understanding are advancing rapidly with novel models and benchmarks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.