Weakly-Supervised RGB-D Salient Object Detection via SAM-driven Pseudo Annotation and State Space Interaction-based Diffusion

2026-07-16 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

A novel scribble-supervised RGB-D Salient Object Detection (SOD) method, named WeakS2Diff, addresses the challenge of sparse pixel-level annotations. This approach integrates a Segment Anything Model (SAM)-driven pseudo annotation generation (SAM-PAG) and a state space interaction-based conditional diffusion model ($S^2$Diff). SAM-PAG expands sparse scribbles into dense pixel-level pseudo annotations by leveraging SAM's dual-branch structure and mask consistency. $S^2$Diff then iteratively refines noisy saliency maps, guided by conditional information. Its core involves a cross-modal conditional generation module, interweaving RGB-D features through frequency integration and implicit-explicit state space interaction, alongside a context injection module to mitigate noise. WeakS2Diff outperforms existing scribble-supervised methods and achieves competitive performance against fully-supervised methods across seven datasets.

Key takeaway

For Computer Vision Engineers aiming to reduce annotation burden in RGB-D Salient Object Detection, this method offers a compelling solution. By leveraging SAM for pseudo-annotation and a diffusion model for refinement, you can achieve high-accuracy SOD with significantly less manual pixel-level labeling. Consider integrating SAM-driven pseudo-annotation and state space interaction-based diffusion techniques into your weakly-supervised SOD pipelines to improve efficiency and performance.

Key insights

A novel weakly-supervised RGB-D SOD method combines SAM-driven pseudo-annotation with a state space interaction-based diffusion model for accurate saliency maps.

Principles

• Sparse scribble annotations can be effectively expanded to dense pixel-level pseudo annotations.
• Diffusion models can iteratively refine noisy saliency maps with conditional guidance.
• Cross-modal feature integration enhances global conditional features for SOD.

Method

The method involves SAM-PAG to generate dense pseudo-annotations from sparse scribbles, followed by $S^2$Diff, a conditional diffusion model that refines these annotations using cross-modal feature integration and context injection.

In practice

• Utilize SAM for expanding sparse annotations in weakly-supervised tasks.
• Apply diffusion models for iterative refinement of noisy segmentation outputs.
• Integrate RGB-D features via frequency and state space interaction for robust conditional guidance.

Topics

• Weakly-Supervised Learning
• Salient Object Detection
• RGB-D Imaging
• Segment Anything Model
• Diffusion Models
• Pseudo Annotation
• Computer Vision

Code references

• Switch457/WeakS2Diff_SOD

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Rarity-Aware Discrete Diffusion with Spatially Consistent Decoding for Photo-Realistic Image Super-Resolution — Discrete diffusion models can achieve photo-realistic SR by explicitly managing token rarity and spatial consistency.
• SAM 3: Building a unified model architecture for detection and tracking — SAM 3 unifies object detection and tracking using distinct transformers for each task.
• Unified Video Dense Prediction from Disjoint Data — UniD unifies eight video dense prediction tasks using disjoint data via diffusion model priors and expert distillation.
• EditSSC: Toward Editable Semantic Occupancy Scenes with Unconditional Diffusion Models — EditSSC repurposes 2D latent diffusion for 3D semantic scene generation, enabling training-free editing capabilities.
• Breaking the Model Forgetting Cycle in Long-Incremental 3D Object Detection — Learning dynamics signals can break the model forgetting cycle in long-incremental 3D object detection.
• Diff-SBSR: Learning Multimodal Feature-Enhanced Diffusion Models for Zero-Shot Sketch-Based 3D Shape Retrieval — Diffusion models, enhanced with multimodal features, can effectively perform zero-shot sketch-based 3D shape retrieval.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.