Contrastive Joint-Embedding Prediction for Representation Learning in Structural MRI

2026-07-12 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

COJEPA is a self-supervised framework designed for representation learning in volumetric brain MRI, addressing the scarcity of labeled medical imaging data. It integrates a joint-embedding predictive architecture (JEPA) with a contrastive loss (CO) to achieve both local predictivity and global discriminability. The model extends I-JEPA to 3D, incorporating foreground-aware block masking, a hierarchical convolutional patch embedding, and world-space sinusoidal positional encodings. Trained without labels on T1-weighted structural MRI from 2,286 individuals (HCP-YA and AABC cohorts, ages 22 to 90), COJEPA was evaluated across zero-shot twin retrieval, BraTS 2024 brain tumor segmentation, and OpenBHB age regression. It achieved a monozygotic twin recall of 0.84 at rank@1 and a finetuning age MAE of 2.55 years on OpenBHB 3.0T, while matching CO on BraTS whole-tumor Dice.

Key takeaway

For Machine Learning Engineers developing models for volumetric brain MRI with scarce labeled data, COJEPA presents a compelling self-supervised solution. Its combined JEPA and contrastive loss objective yields representations that are both discriminative and locally structured, outperforming individual methods in key metrics. You should explore this hybrid approach to enhance model performance in tasks like segmentation, retrieval, and regression, particularly when working with 3D medical images.

Key insights

COJEPA combines JEPA and contrastive learning for robust, self-supervised 3D MRI representation learning.

Principles

• Self-supervised learning mitigates medical data scarcity.
• Combining JEPA and contrastive loss enhances representations.
• Local predictivity and global discriminability are complementary.

Method

Extends I-JEPA to 3D using foreground-aware block masking, hierarchical convolutional patch embedding, and world-space sinusoidal positional encodings.

In practice

• Apply COJEPA for zero-shot twin retrieval.
• Use for brain tumor segmentation (BraTS 2024).
• Implement for age regression tasks (OpenBHB 3.0T).

Topics

• Self-supervised Learning
• Structural MRI
• Joint-Embedding Prediction
• Contrastive Learning
• Brain Tumor Segmentation
• Age Regression

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• How AI Learned to Teach Itself [JEPA] — Self-supervised learning evolves from contrastive methods to JEPA, predicting latent embeddings for robust world models.
• Paper Insights: LeJEPA and LeWorldModel — LeJEPA and LeWorldModel prevent representation collapse by enforcing isotropic Gaussian embedding distributions.
• STST-JEPA: Shallow-Target Spatio-Temporal Joint Embedding Prediction Architecture For EEG Self-Supervised Learning — STST-JEPA effectively predicts brain age and other traits from EEG using a novel self-supervised learning approach.
• MSA-DCNN: A Data-Efficient Multi-Scale Deformable CNN for Medical Image Classification — MSA-DCNN unifies adaptive sampling, multi-scale attention, and self-distillation for data-efficient medical image classification.
• IQ-JEPA: A Joint-Embedding Predictive Architecture with a Hermitian Vision Transformer for Sound Speed and Attenuation Estimation from Ultrasound IQ Data — IQ-JEPA uses a Hermitian Vision Transformer and self-supervision to efficiently estimate ultrasound sound speed and attenuation from IQ data.
• Text-Conditional JEPA for Learning Semantically Rich Visual Representations — Text-Conditional JEPA uses image captions to reduce prediction uncertainty in masked feature prediction for richer visual representations.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.