DDStereo: Efficient Dual Decoder Transformers for Stereo 3D Road Anomaly Detection

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

DDStereo, a novel Dual-Decoder Stereo Transformer, addresses critical safety challenges in stereo-based 3D object detection for autonomous driving: real-time performance and open-set generalization. Existing stereo methods often achieve high accuracy but suffer from slow inference speeds (e.g., S3AD at >80 ms), making them unsuitable for real-time applications. DDStereo features two lightweight decoder branches—one for open-set foreground 2D detection and another for 3D attribute regression—sharing object-level queries for unified target alignment. It achieves real-time performance at 23.5 ms per frame, comparable to monocular approaches, while requiring only 62.65 GFLOPs and 19.6 M parameters. Experiments on KITTI benchmarks show DDStereo achieves state-of-the-art accuracy, including 43.97% AP3D for Car (Moderate) and 74.23% AP3D for unknown OoD objects on KITTI-AR-OoD.

Key takeaway

For Machine Learning Engineers developing autonomous driving systems, DDStereo offers a significant advancement in stereo 3D object detection. If your current stereo solutions struggle with real-time performance or detecting unknown objects, consider adopting DDStereo's dual-decoder Transformer architecture. Its 23.5 ms inference speed and strong open-set generalization on KITTI-AR-OoD mean you can deploy more robust and safer perception systems without relying on expensive LiDAR or compromising on detecting novel obstacles.

Key insights

DDStereo enables real-time open-set 3D object detection from stereo images by decoupling foreground and 3D attribute prediction.

Principles

• Decouple foreground localization from multi-category classification.
• Shared object queries align dual-decoder branches.
• Disparity-aware features enable category-agnostic localization.

Method

DDStereo uses a compact disparity feature extractor and lightweight dual-decoder Transformer. One decoder handles open-set foreground 2D detection, the other 3D attribute regression, both sharing object queries. An object-level depth map is predicted and sampled.

In practice

• Implement dual-decoder architecture for efficiency.
• Utilize shared queries for robust anomaly scoring.
• Employ object-level depth map sampling for 3D localization.

Topics

• Stereo 3D Object Detection
• Open-Set Detection
• Transformer Networks
• Autonomous Driving
• Real-time Inference
• Road Anomaly Detection
• KITTI Benchmark

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Real-Time Source-Free Object Detection — RT-SFOD achieves state-of-the-art source-free object detection by optimizing pseudo-label fusion and multi-scale feature diversification for speed and accuracy.
• DODO: Discrete OCR Diffusion Models — DODO uses block discrete diffusion to achieve faster, accurate OCR by enabling parallel token generation while maintaining structural integrity.
• TRIG: Trajectory-Rig Decoupled Metric Geometry Learning — Decoupling ego-trajectory and camera-rig components in multi-camera systems improves metric geometry and pose estimation for autonomous driving.
• Technical Report for ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge: Leveraging DINOv3 for Robust Outdoor Scene Understanding in Field Robotics — Combining DINOv3, Mask2Former, and test-time augmentation significantly boosts fine-grained semantic segmentation performance in challenging outdoor environments.
• Open-Weather Robust 3D Detection via Dual-Critic Diffusion Alignment — DCDA uses dual-critic guided diffusion to recover degraded LiDAR features for robust 3D detection in unseen weather.
• WildDet3D - an open model for monocular 3D detection — WildDet3D predicts 3D object bounding boxes from single RGB images across 13K+ categories without fine-tuning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.