Propagating Structural Guidance: Synthesizing Fluorescein Angiography from Fundus Images and Sparse OCT Scans

2026-06-15 · Source: Artificial Intelligence · Field: Science & Research — Health & Medical Research, Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition · Depth: Expert, quick

Summary

A novel framework synthesizes Fluorescein Angiography (FFA) images from Color Fundus Photography (CFP) by incorporating structural guidance from Optical Coherence Tomography (OCT). This approach addresses limitations of prior CFP-to-FFA synthesis methods, which relied solely on surface texture and struggled to reconstruct functional vascular information. The researchers constructed the first tri-modally aligned retinal imaging dataset, comprising paired CFP, FFA, and OCT scans from 3,676 patient eyes. Key components include the Spatially Aligned Cross-Modal Fusion (SACMF) module, which projects depth-resolved OCT features onto the fundus plane and injects them into the CFP encoder, and Token-wise Cross-Modality Alignment (TCMA), a contrastive learning strategy aligning CFP and FFA representations. Published on 2026-06-15, the method demonstrates superior synthesis performance and significantly improves downstream disease diagnosis, highlighting its clinical potential as a non-invasive decision-support tool.

Key takeaway

For AI scientists developing non-invasive retinal diagnostic tools, you should consider integrating multi-modal data like OCT scans with fundus photography. This approach, demonstrated by the SACMF and TCMA modules, significantly improves synthesized FFA image quality and downstream disease diagnosis performance. Explore the provided code to adapt this structural guidance framework, enhancing your models' ability to detect subtle vascular abnormalities without invasive procedures.

Key insights

Integrating OCT structural guidance with CFP significantly enhances non-invasive FFA synthesis for improved retinal vascular assessment.

Principles

• Multi-modal data fusion enhances image synthesis.
• Structural guidance improves functional information reconstruction.
• Token-level alignment refines cross-modal representations.

Method

The framework uses a Spatially Aligned Cross-Modal Fusion (SACMF) module to project OCT features onto the fundus plane for CFP encoder injection, complemented by Token-wise Cross-Modality Alignment (TCMA) for spatial representation alignment.

In practice

• Develop non-invasive retinal disease diagnosis tools.
• Enhance FFA synthesis for clinical decision support.
• Utilize multi-modal datasets for medical imaging.

Topics

• Fluorescein Angiography Synthesis
• Retinal Imaging
• Multi-modal Fusion
• Optical Coherence Tomography
• Fundus Photography
• Medical Image Analysis
• Disease Diagnosis

Code references

• while-plus/OCT-guide-FFA-Syn

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• EyeMVP: OCT-Informed Fundus Representation Learning via Paired CFP--OCT Pretraining — EyeMVP uses paired CFP-OCT pretraining to enrich CFP representations with OCT depth information for improved retinal analysis.
• A three-dimensional multi-modal foundation model for optical coherence tomography — OCTCube-M integrates 3D OCT with other retinal imaging modalities for enhanced diagnostic and prognostic capabilities.
• From Baseline to Follow-Up: Counterfactual Spine DXA Image Synthesis in UK Biobank Using a Causal Hierarchical Variational Autoencoder — The CHVAE model synthesizes anatomically plausible, causally consistent spine DXA images by intervening on age, addressing anatomical variation challenges.
• Towards Real-Time Autonomous Navigation: Transformer-Based Catheter Tip Tracking in Fluoroscopy — A multi-threaded deep learning pipeline enables robust, real-time catheter tip tracking in fluoroscopy for autonomous navigation.
• CD-RCM: Generalizable Continuous-Depth Novel View Synthesis for Reflectance Confocal Microscopy — CD-RCM enables continuous-depth, isotropic 3D visualization from anisotropic Reflectance Confocal Microscopy data using a specialized novel-view synthesis model.
• Three-Dimensional Retinal Microvasculature Restoration in OCT Angiography — A deep learning algorithm accurately restores 3D retinal microvasculature from single OCTA volumes, overcoming imaging artifacts.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.