Mask to Concept: Auto-Promptable SAM3 via Efficient Test-Time Concept Embedding Search for Few-Shot Annotation

2026-06-25 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision, Health & Medical Research · Depth: Expert, quick

Summary

Mask to Concept (M2C) is an efficient framework designed to adapt the SAM3 foundation segmentation model for medical few-shot annotation. This approach eliminates the need for external modules, parameter retraining, or manual text engineering. M2C operates by initializing a learnable concept embedding, using it to prompt segmentation, and iteratively updating the embedding through gradient-based minimization of concept segmentation error, all within SAM3's frozen architecture. Furthermore, M2C incorporates a Hybrid Uncertainty Estimation (HUE) module. HUE calculates prediction entropy and maps concept predictions to box prompts to measure concept-geometry prompting inconsistency, flagging highly uncertain samples for human correction. These corrected masks then feed back into M2C, establishing a self-enhancing annotation loop. Experiments on medical segmentation benchmarks demonstrate that M2C achieves state-of-the-art few-shot segmentation performance and outstanding annotation efficiency.

Key takeaway

For Machine Learning Engineers tasked with scaling medical data annotation, M2C offers a pathway to significantly improve efficiency. You should consider integrating this framework to adapt SAM3 for few-shot annotation without extensive retraining or manual text engineering. This approach allows you to leverage a self-enhancing loop with Hybrid Uncertainty Estimation, reducing expert effort and achieving state-of-the-art performance in medical image labeling.

Key insights

M2C adapts SAM3 for few-shot medical annotation by searching for transferable visual concepts within its frozen architecture.

Principles

• Adapting foundation models without retraining.
• Self-enhancing loops improve annotation precision.
• Uncertainty estimation guides human correction.

Method

M2C initializes a learnable concept embedding, prompts SAM3 for segmentation, and updates the embedding via gradients minimizing segmentation error. HUE flags uncertain samples for human feedback.

In practice

• Automate medical image labeling.
• Reduce expert effort in annotation.
• Enhance SAM3 for clinical knowledge.

Topics

• Few-Shot Learning
• Medical Image Segmentation
• SAM3
• Concept Embedding
• Active Learning
• Uncertainty Estimation
• Data Annotation

Code references

• Huster-Hq/M2C

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Entropy-Guided Self-Supervised Learning for Medical Image Classification — Combining ImageNet and entropy-guided MAE pre-training via ensemble significantly boosts medical image classification accuracy and robustness.
• Re-Evaluating Continual Learning with Few-Shot Adaptation — Few-shot evaluation offers a more comprehensive assessment of continual learning model stability and plasticity than traditional 0-shot methods.
• SAM3-LiteText: An Anatomical Study of the SAM3 Text Encoder for Efficient Vision-Language Segmentation — Vision-language segmentation models can achieve efficiency by optimizing text encoders for short, structured prompts.
• Attention-Based Prototype Calibration for Multi-Rater Few-Shot Medical Image Segmentation — Prototype calibration with attention effectively models multi-rater variability in few-shot medical image segmentation.
• A Reproducible Log-Driven AutoML Framework for Interpretable Pipeline Optimization in Healthcare Risk Prediction — Reproducible, log-driven AutoML reveals healthcare risk prediction pipeline component importance and search space redundancy.
• Efficient Search of Implantable Adaptive Cells for Medical Image Segmentation — Early stabilization of operations during differentiable search enables significant acceleration of NAS for medical image segmentation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.