Interpretability of multimodal neural networks for prediction of visual acuity in patients with branch retinal vein occlusion

2026-06-23 · Source: Machine learning : nature.com subject feeds · Field: Health & Wellbeing — Medical Devices & Health Technology, Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Expert, short

Summary

This retrospective proof-of-concept study developed multimodal neural networks to predict 12-month best-corrected visual acuity (BCVA) classes in treatment-naive branch retinal vein occlusion (BRVO) eyes. The best internal model integrated OCT-horizontal scans, OCTA images, baseline BCVA, central subfield thickness, age, and sex. Researchers evaluated performance using adjacent accuracy, exact accuracy, and mean absolute error under five-fold cross-validation, analyzing attribution localization with Pathway Attribution. A supplementary exploratory cross-disease feasibility analysis was also conducted using a diabetic macular edema (DME) OCT cohort with 24-month visual acuity outcomes. While results suggest multimodal imaging and clinical metadata may provide complementary prognostic information for BRVO, the study emphasizes cautious interpretation due to modest predictive performance, its retrospective single-center design, lack of disease-matched external validation, and limited attribution reliability, calling for prospective validation.

Key takeaway

For AI Scientists developing clinical prediction models, you should prioritize prospective, multi-center validation for multimodal neural networks before clinical deployment. Your models, even with interpretability features like Pathway Attribution, require robust external validation to overcome limitations of retrospective, single-center designs. Focus on improving predictive performance beyond modest levels to ensure clinical utility and reliability.

Key insights

Multimodal neural networks can predict 12-month BCVA in BRVO, but require rigorous validation.

Principles

• Multimodal data offers complementary prognostic information.
• Attribution localization aids model interpretability.
• Retrospective studies need prospective validation.

Method

Developed multimodal neural networks to predict 12-month BCVA classes using retinal images and clinical metadata, evaluated via five-fold cross-validation and Pathway Attribution.

In practice

• Combine OCT scans, OCTA, and clinical data for prognosis.
• Use Pathway Attribution for model interpretability.
• Plan prospective studies for clinical validation.

Topics

• Multimodal Neural Networks
• Visual Acuity Prediction
• Branch Retinal Vein Occlusion
• Optical Coherence Tomography
• Explainable AI
• Clinical Prognosis

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• REVEAL: Multimodal Vision-Language Alignment of Retinal Morphometry and Clinical Risks for Incident AD and Dementia Prediction — Jointly modeling retinal images and clinical risk factors via VLM improves early AD and dementia prediction.
• Trusting Right Predictions for Wrong Reasons: A LIME Based Analysis of Deep Learning Interpretability in Lung Cancer Diagnosis — Deep learning models can achieve high predictive accuracy in medical imaging for different reasons, requiring independent interpretability validation.
• A Dual Edge Spatial Jacobian Image Graph for Interpretable Diabetic Retinopathy Grading — Interpretable DR grading benefits from graph representations linking spatial lesion-vessel geometry and embedding-biomarker sensitivity.
• Auditing Multimodal LLM Raters: Central Tendency Bias in Clinical Ordinal Scoring — Multimodal LLMs exhibit a central tendency bias in clinical ordinal scoring, compressing predictions towards the scale's middle.
• Multimodal deep learning model for AI-based functional prognostic risk stratification in patients undergoing radical nephrectomy — A multimodal deep learning model predicts post-surgical kidney function decline in complex renal cell carcinoma patients.
• Multimodal foundation models exploit text to make medical image predictions — Multimodal medical AI models heavily rely on text, which can both enhance and degrade image-based diagnostic accuracy.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine learning : nature.com subject feeds.