Geometry-Aware Uncertainty Coresets for Robust Visual In-Context Learning in Histopathology

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Expert, long

Summary

GAUC is a training-free coreset selection method designed to enhance the reliability of visual in-context learning (ICL) for vision-language models (VLMs) in computational histopathology. It addresses the high sensitivity of ICL to demonstration selection and prompt phrasing by operating directly in the pre-trained multimodal embedding space. GAUC jointly optimizes three objectives: Maximum Mean Discrepancy (MMD) for distributional fidelity, Effective Mutual Information Difference (EMID) for prompt robustness, and a predictive-variance penalty to suppress overconfident outputs. Evaluated on CRC-100K and MHIST datasets with Qwen and LLaVA VLM architectures, GAUC consistently improved accuracy (e.g., 0.610 on CRC-100K), calibration (reducing ECE from 0.153 to 0.145), and prompt robustness over existing ICL selection methods and dataset-distillation baselines, all without requiring any gradient updates.

Key takeaway

For AI Scientists and ML Engineers deploying VLMs in clinical settings, GAUC offers a critical solution for reliable in-context learning. You should consider integrating this training-free coreset selection to mitigate prompt sensitivity and overconfident predictions, ensuring more accurate and robust diagnostic support. This approach enhances VLM trustworthiness without costly fine-tuning, making it suitable for privacy-sensitive medical data.

Key insights

GAUC enables robust, training-free visual in-context learning for VLMs in histopathology by optimizing coreset selection.

Principles

• Preserve global data structure via MMD.
• Ensure prompt robustness with EMID.
• Suppress overconfident predictions.

Method

GAUC selects a query-independent coreset by jointly minimizing Maximum Mean Discrepancy, Effective Mutual Information Difference, and predictive variance in the VLM's frozen embedding space, without gradient updates.

In practice

• Improve VLM diagnostics in histopathology.
• Enhance ICL reliability in safety-critical domains.

Topics

• Histopathology
• In-Context Learning
• Vision-Language Models
• Coreset Selection
• Prompt Robustness
• Predictive Calibration

Code references

• fx-erick/GAUC

Best for: Computer Vision Engineer, AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• Geometry-Aware Uncertainty Coresets for Robust Visual In-Context Learning in Histopathology — GAUC improves VLM in-context learning for histopathology by selecting robust coresets without parameter updates.
• Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence — Self-function vectors enable robust aleatoric and epistemic uncertainty decomposition in LLM In-Context Learning.
• DualGate-Net: A Prior-Gated Dual-Encoder Framework for Histopathology Cell Detection — Adaptive, gated fusion of local and global features with contextual priors improves histopathology cell detection.
• Parameter-Efficient Subspace Decoupling ViT for Mitigating Multi-Task Negative Transfer in Histological Scoring — Subspace-decoupled ViT with orthogonal adapters mitigates negative transfer in multi-task histological scoring, improving efficiency.
• Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming — A novel VLM framework integrates lesion-focused reasoning and uncertainty-aware rewards for improved ultrasound diagnosis.
• E-MRL: Cross-view Aligned Evidence-driven Multimodal Reinforcement Learning for Reliable 3D Tumor Analysis — Grounding Vision-Language Models in specific visual evidence via reinforcement learning significantly reduces hallucinations in medical report generation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.