HPR-SAM: Hierarchical Probabilistic Representation Learning for Prompt-free SAM-based Medical Image Segmentation

2026-07-08 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition, Health & Medical Research · Depth: Expert, quick

Summary

HPR-SAM, or Hierarchical Probabilistic Representation Learning, introduces a novel framework for prompt-free medical image segmentation using the Segment Anything Model (SAM). This approach addresses the limitation of existing methods that focus on prompt generation but neglect the expressiveness of anatomical representations, which often fail to capture global anatomical priors, intra-structure diversity, and local structural reliability. HPR-SAM learns complementary anatomical representations through three components: Distributional Anatomical Representation (DAR), Multi-component Anatomical Representation (MAR), and Local Reliability Representation (LRR). It then integrates their predictions using Hierarchical Prediction Fusion (HPF), maintaining compatibility with the original SAM decoder. Experiments on the Synapse, LA, and PROMISE12 datasets demonstrate that HPR-SAM achieves state-of-the-art performance on Synapse and the best performance in few-shot settings on LA and PROMISE12.

Key takeaway

For medical imaging researchers and machine learning engineers developing automated segmentation tools, HPR-SAM offers a significant advancement in prompt-free SAM adaptation. If your current methods struggle with capturing diverse anatomical features or achieving high accuracy in few-shot scenarios, you should consider exploring hierarchical probabilistic representation learning. This approach can improve segmentation performance on complex medical datasets, potentially reducing the need for extensive manual prompting and enhancing model robustness.

Key insights

HPR-SAM improves prompt-free medical image segmentation by learning hierarchical probabilistic anatomical representations for SAM.

Principles

• Anatomical representation expressiveness limits prompt quality.
• Probabilistic representations capture diverse anatomical features.
• Hierarchical fusion integrates complementary representations.

Method

The HPR framework learns DAR, MAR, and LRR for complementary anatomical representations. These are then integrated via Hierarchical Prediction Fusion (HPF) with the SAM decoder.

In practice

• Enhance SAM for medical image segmentation.
• Improve few-shot learning in medical imaging.
• Address diverse anatomical structure segmentation.

Topics

• Medical Image Segmentation
• Segment Anything Model
• Prompt-free Learning
• Probabilistic Representations
• Hierarchical Prediction Fusion
• Few-shot Learning

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Mask to Concept: Auto-Promptable SAM3 via Efficient Test-Time Concept Embedding Search for Few-Shot Annotation — M2C adapts SAM3 for few-shot medical annotation by searching for transferable visual concepts within its frozen architecture.
• SIAM: Head and Brain MRI Segmentation from Few High-Quality Templates via Synthetic Training — SIAM enables robust, contrast-agnostic 3D head and brain MRI segmentation with minimal high-quality templates.
• ROBUST-WT: Robust Uncertainty-aware Segmentation Transform via Whitening and Training Enhancements — ROBUST-WT demonstrates that targeted training enhancements can substantially improve medical image segmentation robustness without altering core architecture.
• Learning a Sampling-Free Variational DNN Plugin from Tiny Training Sets to Refine OOD Segmentation With Uncertainty Estimation — VarDeepPCA refines OOD medical image segmentations by learning anatomical geometric priors from small in-distribution datasets using sampling-free variational learning.
• A 3D SAM-Based Progressive Prompting Framework for Multi-Task Segmentation of Radiotherapy-induced Normal Tissue Injuries in Limited-Data Settings — A progressive prompting framework enhances 3D SAM for accurate, multi-task segmentation of small, sparse radiotherapy injuries with limited data.
• Preserve the Hard, Regenerate the Rest: Uncertainty-Guided Synthetic Training Data Augmentation with Diffusion Models — Improve semantic segmentation by augmenting uncertain regions with diffusion models, preserving original labels.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.