SUP-MCRL: Subject-aware Unified Pseudo-feature Coded Multimodal Contrastive Representation Learning for EEG Visual Decoding

2026-06-15 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition · Depth: Expert, quick

Summary

SUP-MCRL is a novel subject-aware unified pseudo-feature coded multimodal contrastive representation learning framework designed to improve neural visual decoding from non-invasive brain-computer interfaces, particularly for natural visual experiences. It addresses the fidelity degradation and spurious zero-shot alignment issues found in conventional methods that often neglect semantic consistency and subject selectivity. The framework integrates three key mechanisms: a Semantic-entity Aware Visual Encoder (SAVE) for semantic content extraction via spatial attention, a Unified EEG Enhancer (UEE) employing multi-scale atrous convolutions and inter-band attention for adaptive cross-subject robustness, and a Prototype-based Progressive Augmenter (PPA) that uses an EMA-updated pseudo-feature pool to prevent representation collapse. Zero-shot experiments on the THINGS-EEG dataset demonstrate superior performance, achieving 66.0%/91.9% (Top-1/Top-5) intra-subject and 24.0%/52.9% LOSO accuracy, outperforming existing methods.

Key takeaway

For Research Scientists developing non-invasive brain-computer interfaces, if you are struggling with fidelity degradation and generalization to natural visual experiences, consider integrating SUP-MCRL's mechanisms. You should explore its Semantic-entity Aware Visual Encoder for semantic content, the Unified EEG Enhancer for cross-subject robustness, and the Prototype-based Progressive Augmenter to prevent representation collapse. This approach offers significantly improved zero-shot accuracy, as demonstrated by 66.0%/91.9% intra-subject and 24.0%/52.9% LOSO results on THINGS-EEG.

Key insights

SUP-MCRL enhances EEG visual decoding by integrating semantic, subject-adaptive, and collapse-prevention mechanisms for robust multimodal representation learning.

Principles

• Semantic consistency is crucial for robust multimodal representation learning.
• Subject selectivity improves generalization in neural decoding.

Method

SUP-MCRL integrates SAVE for semantic attention, UEE for cross-subject robustness via multi-scale atrous convolutions and inter-band attention, and PPA with an EMA-updated pseudo-feature pool to prevent representation collapse.

In practice

• Apply spatial attention for semantic content extraction.
• Utilize EMA-updated pseudo-feature pools to stabilize training.

Topics

• EEG Visual Decoding
• Brain-Computer Interfaces
• Multimodal Contrastive Learning
• Semantic Attention
• Cross-Subject Robustness
• Representation Collapse

Code references

• NZWANG/SUP-MCRL

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Neural Visual Decoding via Cognitive guided Adaptive Blurring and Information Constrained Alignment — CAIA enhances EEG-based visual decoding by adaptively blurring images and filtering EEG signals based on cognitive principles.
• GAZE: Grounded Agentic Zero-shot Evaluation with Viewer-Level Tools and Literature Retrieval on Rare Brain MRI — Agentic VLMs with iterative tool use and retrieval enhance medical image interpretation, especially for rare conditions.
• Learning aligned EEG representations with subject-specific encoders — Hybrid EEG encoders learn subject alignment, mitigating inter-subject distribution shifts.
• Seeing the imagined: a latent functional alignment in visual imagery decoding from fMRI data — Latent functional alignment significantly improves fMRI-based visual imagery decoding by leveraging perception-learned semantic structures.
• UF-AMA: A unified framework for cross-domain emotion recognition via adaptive multimodal alignment — UF-AMA unifies multimodal physiological signal fusion with adaptive alignment and multi-level domain adaptation for robust cross-domain emotion recognition.
• KAST-BAR: Knowledge-Anchored Semantically-Dynamic Topology Brain Autoregressive Modeling for Universal Neural Interpretation — KAST-BAR aligns multi-level brain topology with expert semantic space to enhance EEG foundation models.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.