NegROI: Click-Centric Uncertainty-Guided Refinement with Scene-Conditioned Negative Prompts for Robust Interactive 3D Segmentation

2026-07-07 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

NegROI is a novel transformer-based interactive framework designed for robust 3D segmentation in point clouds, addressing limitations of existing methods such as coarse voxel resolution and persistent false positives. It tackles challenges posed by density and scale shifts across datasets like RGB-D reconstructions and LiDAR scans. The framework employs click-centric multi-resolution refinement, focusing on a local Region Of Interest (ROI) around user clicks on a finer grid and integrating refined logits into the coarse mask. To enhance efficiency and robustness, NegROI introduces uncertainty-driven selective refinement, prioritizing ambiguous areas. It also models difficult background patterns using scene-conditioned negative prompts, which are stabilized by a diversity regularizer. Furthermore, boundary-aware hard negative mining supervises negative-prompt attention towards high-confidence false positives near boundaries. Experiments on ScanNet, S3DIS, and KITTI demonstrate improved click efficiency, fewer false positives, and superior cross-dataset robustness over current state-of-the-art baselines.

Key takeaway

For Machine Learning Engineers developing interactive 3D segmentation systems, especially when facing challenges with fine boundaries or persistent false positives across diverse datasets, NegROI presents a robust solution. You should consider integrating its click-centric multi-resolution refinement and scene-conditioned negative prompts. This method improves click efficiency and reduces false positives, offering stronger cross-dataset robustness than current baselines.

Key insights

NegROI refines 3D segmentation using click-centric multi-resolution processing and scene-conditioned negative prompts for robustness across diverse datasets.

Principles

• Refine only local ROI for efficiency.
• Prioritize ambiguous regions via uncertainty.
• Model hard backgrounds with negative prompts.

Method

NegROI refines coarse voxel predictions by focusing on a local ROI around clicks on a finer grid, fusing logits. It uses uncertainty-driven selective refinement and scene-conditioned negative prompts with diversity regularization and boundary-aware hard negative mining.

In practice

• Apply multi-resolution refinement locally.
• Use uncertainty to guide refinement.
• Incorporate negative prompts for background.

Topics

• 3D Segmentation
• Point Clouds
• Transformer Models
• Interactive Segmentation
• Negative Prompts
• Multi-resolution Refinement

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Texture-Shape Bias Balancing for Robust Synthetic-to-Real Semantic Segmentation in Automotive NIR Imagery — A generative augmentation framework reduces the synthetic-to-real domain gap for automotive NIR semantic segmentation by balancing texture-shape bias.
• NG-GS: NeRF-Guided 3D Gaussian Splatting Segmentation — NG-GS enhances 3D Gaussian Splatting segmentation by integrating NeRF-guided continuous feature fields to resolve boundary artifacts.
• Domain-Guided Prompting of the Segment Anything Model for Seismic Interpretation: The Role of Attributes, Visualization, and Hybrid Prompts — Zero-shot SAM adaptation for seismic interpretation is achieved by combining domain-guided visualization with hybrid prompting.
• CMTM: Cross-Modal Token Modulation for Unsupervised Video Object Segmentation — CMTM enhances unsupervised video object segmentation by deeply integrating appearance and motion cues via token modulation.
• Preserve the Hard, Regenerate the Rest: Uncertainty-Guided Synthetic Training Data Augmentation with Diffusion Models — Improve semantic segmentation by augmenting uncertain regions with diffusion models, preserving original labels.
• Advanced SAM 3: Multi-Modal Prompting and Interactive Segmentation — SAM 3 offers flexible multi-modal and interactive prompting for precise, context-aware image segmentation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.