EyeMVP: OCT-Informed Fundus Representation Learning via Paired CFP--OCT Pretraining

2026-06-13 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Medical Imaging AI · Depth: Expert, quick

Summary

EyeMVP is a novel cross-modal retinal foundation model designed to enhance color fundus photography (CFP) representations with depth-resolved information from optical coherence tomography (OCT). It was pretrained on 674,893 same-eye same-day paired CFP-OCT image triples from 112,642 patients across eight hospitals in China. The model employs cross-modal masked reconstruction, using source-constrained cross-attention and CFP-derived structural masks to align non-aligned imaging geometries. EyeMVP requires only CFP images for inference, making it suitable for screening. Across 16 downstream tasks, including classification and segmentation, EyeMVP consistently outperformed other retinal foundation models, particularly on tasks involving macular and optic nerve structure. For CFP-challenging macular diseases, it achieved an AUROC of 0.948 for macular edema, significantly higher than EyeCLIP's 0.852, and 0.825 for myopic macular schisis. An exploratory reader study indicated EyeMVP surpassed junior and intermediate ophthalmologists on macular edema and showed numerically higher balanced accuracy than all reader groups on myopic macular schisis.

Key takeaway

For AI Scientists developing diagnostic tools for ophthalmology, EyeMVP demonstrates a powerful method to enhance unimodal screening. Its approach of integrating OCT-informed representations into CFP models significantly boosts diagnostic accuracy for complex macular diseases like macular edema and myopic macular schisis. You should explore cross-modal masked reconstruction and source-constrained cross-attention to improve existing CFP-based models, particularly for conditions requiring depth information. This offers a practical route to stronger, more accurate retinal analysis in screening settings.

Key insights

EyeMVP uses paired CFP-OCT pretraining to enrich CFP representations with OCT depth information for improved retinal analysis.

Principles

• Pixel-level cross-modal reconstruction enriches CFP with OCT supervision.
• Cross-attention with structural masks accommodates non-aligned imaging.
• Paired CFP-OCT pretraining improves performance on macular/optic nerve tasks.

Method

EyeMVP uses cross-modal masked reconstruction with source-constrained cross-attention and CFP-derived structural masks, pretrained on paired CFP-OCT image triples. It requires only CFP for inference.

In practice

• Enhance CFP-based retinal analysis in screening settings.
• Improve diagnosis of CFP-challenging macular diseases.
• Potentially aid junior ophthalmologists in specific diagnoses.

Topics

• EyeMVP
• Cross-modal Learning
• Retinal Screening
• Optical Coherence Tomography
• Color Fundus Photography
• Macular Disease Diagnosis

Best for: Computer Vision Engineer, AI Scientist, Research Scientist

Related on AIssential

• Propagating Structural Guidance: Synthesizing Fluorescein Angiography from Fundus Images and Sparse OCT Scans — Integrating OCT structural guidance with CFP significantly enhances non-invasive FFA synthesis for improved retinal vascular assessment.
• A three-dimensional multi-modal foundation model for optical coherence tomography — OCTCube-M integrates 3D OCT with other retinal imaging modalities for enhanced diagnostic and prognostic capabilities.
• MPD$^2$-Router: Mask-aware Multi-expert Prior-regularized Dual-head Deferral Router in Glaucoma Screening and Diagnosis — MPD$^2$-Router optimizes glaucoma screening by routing complex cases to available human experts, balancing cost and diagnostic accuracy.
• Parameter-Efficient Subspace Decoupling ViT for Mitigating Multi-Task Negative Transfer in Histological Scoring — Subspace-decoupled ViT with orthogonal adapters mitigates negative transfer in multi-task histological scoring, improving efficiency.
• Computational Imaging Priors for Wireless Capsule Endoscopy: Monte Carlo-Guided Hemoglobin Mapping for Rare-Anomaly Detection — A Monte Carlo-inspired prior improves wireless capsule endoscopy classification by better distinguishing hemoglobin from confounding factors.
• Three-Dimensional Retinal Microvasculature Restoration in OCT Angiography — A deep learning algorithm accurately restores 3D retinal microvasculature from single OCTA volumes, overcoming imaging artifacts.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.