A 3D SAM-Based Progressive Prompting Framework for Multi-Task Segmentation of Radiotherapy-induced Normal Tissue Injuries in Limited-Data Settings

2026-04-16 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Medical Image Analysis · Depth: Expert, extended

Summary

A new 3D SAM-based progressive prompting framework has been developed for multi-task segmentation of radiotherapy-induced normal tissue injuries, such as osteoradionecrosis (ORN), cerebral edema (CE), and cerebral radiation necrosis (CRN). This framework addresses challenges like scarce voxel-level annotations and the small, sparse nature of lesions in 3D images. It integrates three types of prompts: text prompts for task conditioning using clinical and demographic information, dose-guided box prompts derived from high-dose radiotherapy regions for coarse localization, and automatically generated click prompts for iterative boundary refinement. Additionally, a small-target focus loss is employed to concentrate optimization on relevant, high-dose regions. Evaluated on a curated dataset of 70 head-and-neck radiotherapy cases, the method consistently outperforms five state-of-the-art segmentation approaches, achieving a Dice score of 77.11%, IoU of 63.23%, and significantly lower HD95 (5.70) and ASSD (1.39).

Key takeaway

For Computer Vision Engineers developing medical image segmentation models for rare conditions with limited data, this framework offers a robust approach. You should consider integrating progressive prompting strategies, including text-based task conditioning, dose-guided spatial priors, and iterative click-based refinement, into your SAM-based architectures. This can significantly improve segmentation accuracy and boundary delineation for small, sparse lesions, outperforming generic segmentation models.

Key insights

A progressive prompting framework enhances 3D SAM for accurate, multi-task segmentation of small, sparse radiotherapy injuries with limited data.

Principles

• Combine large-scale pretraining with task-specific prompts.
• Progressive prompting improves localization and refinement.
• Focus loss on high-dose regions addresses target imbalance.

Method

The framework uses a 3D SAM backbone adapted with text prompts for task conditioning, dose-guided box prompts for coarse localization, and iterative click prompts for boundary refinement, all optimized with a small-target focus loss.

In practice

• Curate dedicated datasets for rare medical conditions.
• Utilize clinical priors (e.g., dose maps) for prompt generation.
• Employ iterative refinement with simulated clicks during training.

Topics

• Radiotherapy-induced Injury Segmentation
• 3D SAM
• Progressive Prompting
• Dose-guided Box Prompts
• Click Simulation

Best for: Computer Vision Engineer, AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• A 3D SAM-Based Progressive Prompting Framework for Multi-Task Segmentation of Radiotherapy-induced Normal Tissue Injuries in Limited-Data Settings — A 3D SAM-based framework improves radiotherapy injury segmentation using progressive prompting and a small-target focus loss.
• SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentation — SegMoTE adapts SAM for medical image segmentation using token-level experts and progressive prompting, achieving high performance with minimal data.
• Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors — RAD3D-Prefix enables efficient 3D CT report generation by adapting frozen LLMs with diagnostic priors.
• SAM 3 - AI at Meta — SAM 3 offers state-of-the-art object segmentation and tracking across images and videos using diverse text and visual prompts.
• Single-Stage Hierarchical Rectification for Weakly Supervised Histopathology Segmentation — SSHR proactively purifies features in a single stage, outperforming multi-stage WSSS and reducing training time significantly.
• From Boundaries to Semantics: Prompt-Guided Multi-Task Learning for Petrographic Thin-section Segmentation — Petro-SAM enables high-quality joint grain-edge and lithology segmentation in petrographic images by adapting SAM.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.