Boosting Brain-to-Image Decoding with TRIBE v2 Data Augmentation

2026-06-04 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computational Neuroscience · Depth: Expert, quick

Summary

Brain-to-image decoding, often constrained by limited labeled neural data, can be significantly improved by augmenting small fMRI datasets with synthetic data. Researchers investigated using TRIBE v2, a large encoding model pretrained on over 1000 hours of fMRI responses to video, audio, and language, to generate this synthetic data. Experiments conducted on two datasets, the 7T fMRI Natural Scenes Dataset and 3T fMRI BOLD5000, demonstrated up to a 68% improvement in Top-10 image-retrieval accuracy compared to decoders trained solely on real data. The study found that the optimal proportion of augmented data varies depending on the data source. Furthermore, image decoders trained exclusively on synthetic fMRI data performed above chance in some configurations, suggesting TRIBE v2 supports zero-shot brain-to-image decoding. These findings highlight the potential of large-scale fMRI response models to enhance data efficiency for image decoding.

Key takeaway

For research scientists developing brain-to-image decoding models with scarce fMRI data, you should integrate synthetic data augmentation using large pretrained encoding models like TRIBE v2. This approach can yield up to a 68% improvement in image-retrieval accuracy, potentially enabling zero-shot decoding. Carefully adjust the synthetic data proportion based on your specific fMRI source to optimize performance and overcome data limitations.

Key insights

Synthetic fMRI data from TRIBE v2 significantly boosts brain-to-image decoding accuracy, enabling zero-shot capabilities.

Principles

• Data augmentation improves fMRI decoding.
• Pretrained models generate effective synthetic data.
• Zero-shot decoding is possible with synthetic fMRI.

Method

Augment small fMRI datasets with synthetic data from TRIBE v2, a large encoding model pretrained on 1000+ hours of fMRI responses to video, audio, and language.

In practice

• Use TRIBE v2 for fMRI data synthesis.
• Adjust synthetic data proportion per source.
• Explore zero-shot brain decoding applications.

Topics

• Brain-to-Image Decoding
• fMRI Data Augmentation
• TRIBE v2
• Encoding Models
• Zero-Shot Learning
• Neural Data Scarcity

Best for: AI Scientist, Research Scientist

Related on AIssential

• Boosting Brain-to-Image Decoding with TRIBE v2 Data Augmentation — Synthetic fMRI data from pretrained models like TRIBE v2 can significantly boost brain-to-image decoding in low-data regimes.
• Today we're introducing TRIBE v2 (Trimodal Brain Encoder), a foundation model trained to predict how the human brain responds to almost any sight or sound. Building on our Algonauts 2025 award-winning architecture, TRIBE v2 draws on 500+ hours of fMR - x.com — TRIBE v2 is a foundation model predicting brain responses to sights and sounds using fMRI data.
• Zero-Shot Captioning for Cultural Heritage: Automated Image Analysis of Traditional Indonesian Clothing — Retrieval-augmented vision-language models effectively generate culturally accurate captions for low-resource heritage data.
• Noise-Aware Visual Representation Learning for Medical Visual Question Answering — Denoising visual embeddings before LLM projection significantly improves Med-VQA robustness against input noise.
• A foundation model of vision, audition, and language for in-silico neuroscience | Research - AI at Meta — TRIBE v2 unifies cognitive neuroscience by predicting human brain activity across diverse conditions using a tri-modal foundation model.
• Introducing TRIBE v2: A Predictive Foundation Model Trained to Understand How the Human Brain Processes Complex Stimuli - AI at Meta — TRIBE v2 is a foundation model predicting high-resolution human brain activity from diverse sensory stimuli.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.