Geometry-Aware Uncertainty Coresets for Robust Visual In-Context Learning in Histopathology

2026-05-18 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition, Health & Medical Research · Depth: Expert, quick

Summary

A new training-free coreset selection method, GAUC (Geometry-Aware Uncertainty Coresets), has been developed to improve visual in-context learning (ICL) in computational histopathology using vision-language models (VLMs). VLMs are promising for clinical reasoning but face challenges with fine-tuning on limited expert data and ICL's sensitivity to example selection. GAUC addresses these issues by operating directly in the pre-trained multimodal embedding space, optimizing for distributional fidelity via Maximum Mean Discrepancy, bounding performance degradation from prompt paraphrases using an Effective Mutual Information Difference regularizer, and suppressing overconfident outputs with a predictive-variance penalty. Tested on CRC-100K and MHIST datasets with various open-source VLM architectures, GAUC consistently enhanced accuracy, calibration, and prompt robustness compared to existing ICL selection methods and dataset-distillation baselines, all without requiring any gradient updates.

Key takeaway

For AI scientists and research scientists developing VLM applications in histopathology, GAUC offers a training-free method to significantly improve in-context learning performance. You should consider integrating GAUC into your VLM workflows to enhance diagnostic accuracy, calibration, and robustness against prompt variations, thereby reducing the need for costly fine-tuning on scarce expert data.

Key insights

GAUC improves VLM in-context learning for histopathology by selecting robust coresets without parameter updates.

Principles

• Multimodal embedding space is key for robust VLM ICL.
• Distributional fidelity improves coreset representativeness.
• Prompt robustness requires joint vision-text alignment.

Method

GAUC jointly optimizes Maximum Mean Discrepancy for distributional fidelity, Effective Mutual Information Difference for prompt robustness, and a predictive-variance penalty for output stability, all within the VLM's pre-trained multimodal embedding space.

In practice

• Apply GAUC for VLM ICL in histopathology.
• Use GAUC to enhance diagnostic reliability.
• Improve prompt robustness in VLM applications.

Topics

• Geometry-Aware Uncertainty Coresets
• Visual In-Context Learning
• Histopathology
• Vision-Language Models
• Multimodal Embedding Space

Best for: AI Scientist, Research Scientist

Related on AIssential

• MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models — Biomedical VLMs exhibit pervasive perceptual weaknesses, especially in object detection and open-ended tasks, despite medical adaptation.
• Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs — Fine-grained preference optimization improves medical LVLM accuracy by addressing token-level errors and visual grounding.
• Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA — MLLM uncertainty calibration in Medical VQA improves via a composite loss finetuning framework.
• OphIn-500K: Curating Web-Scale Visual Instructions for Scaling Ophthalmic Multimodal Large Language Models — Curating web-scale ophthalmic video data via a specialized pipeline enables superior domain-specific Multimodal Large Language Models.
• Learning Uncertainty from Sequential Internal Dispersion in Large Language Models — Uncertainty in LLMs can be reliably inferred from the dispersion of hidden states across layers and token sequences.
• DualGate-Net: A Prior-Gated Dual-Encoder Framework for Histopathology Cell Detection — Adaptive, gated fusion of local and global features with contextual priors improves histopathology cell detection.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.