Towards Interpretable Foundation Models for Retinal Fundus Images

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Expert, long

Summary

Dual-IFM is an interpretable foundation model designed for retinal fundus images, addressing the critical lack of transparency in high-stakes medical AI. It provides both local interpretability through class evidence maps and global interpretability via a 2D projection layer that visualizes the model's representation space. The model was trained on over 800,000 color fundus photography images from EyePACS, AREDS, and UKBiobank datasets. Dual-IFM achieved performance comparable to state-of-the-art foundation models like RETFound, which has up to 16x more parameters (303M vs. 18.3M), while offering faster inference (157 ms vs. 322 ms). It demonstrated strong performance on Diabetic Retinopathy (AUROC: 0.822±0.054) and Age-related Macular Degeneration (AUROC: 0.928±0.004) tasks, maintaining generalizable features on out-of-distribution data.

Key takeaway

For AI Scientists developing medical imaging solutions, you should prioritize inherently interpretable foundation models like Dual-IFM. This approach offers both local class evidence maps and global representation space visualization, crucial for clinical trust and deployment. You can achieve competitive performance with significantly fewer parameters and faster inference compared to larger, less interpretable models. Consider integrating similar dual interpretability designs to enhance model transparency and robustness in high-stakes domains.

Key insights

Foundation models for medical imaging can achieve high performance and inherent interpretability simultaneously.

Principles

• Dual interpretability enhances trust in medical AI.
• Self-supervised pretraining improves downstream task performance.
• Parametric 2D projection enables direct visualization.

Method

Dual-IFM uses a BagNet architecture with the t-SimCNE algorithm for self-supervised pretraining. It learns a parametric mapping to a 2D space for global interpretability and generates class evidence maps for local interpretability.

In practice

• Use BagNet architecture for inherent local interpretability.
• Implement a parametric 2D projection layer for global visualization.
• Apply sparsity constraints during fine-tuning for localized evidence maps.

Topics

• Foundation Models
• Medical Imaging AI
• Retinal Fundus Images
• Model Interpretability
• Self-Supervised Learning
• BagNet Architecture

Code references

• berenslab/interpretable_FM

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Vision Foundation Models in Radiology: A Scoping Review of Data, Methodology, Evaluation and Clinical Translation — Radiology-specific Vision Foundation Models show promising transferability but face significant hurdles for clinical translation.
• A Dual Edge Spatial Jacobian Image Graph for Interpretable Diabetic Retinopathy Grading — Interpretable DR grading benefits from graph representations linking spatial lesion-vessel geometry and embedding-biomarker sensitivity.
• EFIQA: Explainable Fundus Image Quality Assessment via Anatomical Priors — EFIQA assesses fundus image quality by learning anatomical correctness rather than degradation, enabling label-free, explainable spatial mapping.
• SAIL: Structure-Aware Interpretable Learning for Anatomy-Aligned Post-hoc Explanations in OCT — Integrating anatomical priors and semantic features improves deep learning explainability in medical imaging.
• EyeMVP: OCT-Informed Fundus Representation Learning via Paired CFP--OCT Pretraining — EyeMVP uses paired CFP-OCT pretraining to enrich CFP representations with OCT depth information for improved retinal analysis.
• A universal foundation model for grounded biomedical image interpretation — UniBiomed unifies diagnostic findings and target segmentation in biomedical images, enhancing AI interpretability for clinical use.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.