TCLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models

2026-07-10 · Source: Computer Vision and Pattern Recognition · Field: Health & Wellbeing — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Expert, quick

Summary

TCLA, a novel training-free few-shot adaptation method, addresses the performance decline of Medical Vision-Language Models (VLMs) on out-of-distribution (OOD) data. Medical VLMs often suffer from domain shifts and class bias inherited from large-scale pretraining, and existing few-shot adaptation techniques can be unstable in extremely low-data regimes like 1-shot scenarios. TCLA operates by correcting inference logits using a small set of support samples, which enhances inter-class deconfusion and mitigates domain shift. This method is fast, model-agnostic, and requires no additional trainable components. Extensive experiments across nine datasets, encompassing X-ray, Ultrasound, MRI, CT, and Histopathology modalities, demonstrate that TCLA consistently improves OOD performance for Medical VLMs. In most cases, it surpasses the effectiveness of existing training-based adaptation methods.

Key takeaway

For Research Scientists developing Medical Vision-Language Models, TCLA offers a robust solution for improving out-of-distribution performance without complex retraining. If you are struggling with VLM instability in low-data regimes or domain shifts across medical imaging modalities, consider integrating TCLA. This training-free, model-agnostic approach can significantly boost your model's effectiveness on new datasets, especially when few-shot adaptation is critical.

Key insights

TCLA is a training-free, few-shot logit adaptation method that improves Medical VLM performance on OOD data by correcting inference logits.

Principles

• OOD performance in VLMs suffers from domain shifts.
• Class bias from pretraining impacts VLM effectiveness.
• Training-free adaptation can enhance robustness.

Method

TCLA corrects Medical VLM inference logits using a small set of support samples. This process improves inter-class deconfusion and reduces domain shift without introducing trainable components.

In practice

• Apply TCLA for OOD medical image classification.
• Use TCLA in low-data (e.g., 1-shot) scenarios.
• Evaluate TCLA across diverse medical modalities.

Topics

• Medical Vision-Language Models
• Few-Shot Learning
• Out-of-Distribution Generalization
• Logit Adaptation
• Training-Free Methods
• Medical Imaging

Best for: Computer Vision Engineer, AI Scientist, Research Scientist

Related on AIssential

• Leveraging Offline Supervision for Efficient and Generalizable Reinforcement Learning in Large-Scale Vision-Language-Action Models — Offline supervision significantly boosts RL training efficiency for VLA models while preserving strong out-of-distribution generalization.
• Decoupling Language Guidance from Backbones for Text-Guided Medical Segmentation — BTHA decouples language guidance from backbones for flexible text-guided medical segmentation.
• MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration — Logit miscalibration in merged VLM experts degrades multi-domain performance; MED-DSLC restores global comparability through domain supervision and logit scaling.
• Training-Free Test-Time Contrastive Learning for Large Language Models — TF-TTCL improves LLM robustness under distribution shift by distilling self-generated inference experiences into explicit textual rules.
• OpenMedQ: Broad Open Pretraining for Medical Vision-Language Models — Broad open pretraining on diverse medical data enables SOTA performance with smaller models.
• Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generation — A new Vietnamese PET/CT dataset and clinical benchmarks significantly enhance medical VLM performance, especially for low-resource languages.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.