fMRI2Face: A Full-HD fMRI-Video Dataset and Geometry-Guided Neural Decoding Framework for Dynamic Human Face Reconstruction

2026-07-24 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Neuroscience & Brain-Computer Interfaces · Depth: Expert, quick

Summary

fMRI2Face is a novel geometry-guided neural video decoding framework designed to reconstruct dynamic human faces from fMRI signals. It is introduced alongside fMRI-Face, the first fMRI dataset paired with controllable full-HD digital human facial videos rendered at 1920x1080 resolution. This dataset comprises 62,856 paired fMRI-video samples, recorded while participants watched photorealistic facial videos with controlled identity, expression, and head pose. The fMRI2Face framework utilizes Brain-derived Appearance Context for global identity attributes and Morphable 3D Facial Control for explicit geometry-aware guidance, integrating these via Neural-Controlled Video Diffusion with auxiliary latent completion. Experiments demonstrate fMRI2Face's improved fidelity, identity preservation, facial geometry, and motion consistency compared to baseline methods.

Key takeaway

For Research Scientists developing brain-computer interfaces or neural decoding models, fMRI2Face offers a significant advancement in reconstructing complex visual stimuli. You should explore integrating geometry-guided controls and high-resolution datasets like fMRI-Face into your own frameworks to improve reconstruction fidelity and preserve identity and motion dynamics. This approach provides a robust benchmark for future fMRI-based digital human reconstruction efforts.

Key insights

A new fMRI dataset and geometry-guided neural framework enable high-fidelity reconstruction of dynamic human faces from brain activity.

Principles

• Complementary neural controls enhance reconstruction.
• Explicit geometry guidance improves facial dynamics.
• High-resolution datasets are crucial for decoding.

Method

fMRI2Face derives Brain-derived Appearance Context and Morphable 3D Facial Control from fMRI, integrating them through Neural-Controlled Video Diffusion with auxiliary latent completion to reconstruct facial videos.

In practice

• Study dynamic face perception mechanisms.
• Benchmark fMRI-based digital human reconstruction.

Topics

• fMRI
• Face Reconstruction
• Neural Decoding
• Video Diffusion
• Brain-Computer Interface
• Neuroscience Dataset

Best for: AI Scientist, Research Scientist, Computer Vision Engineer

Related on AIssential

• fMRI2Face: A Full-HD fMRI-Video Dataset and Geometry-Guided Neural Decoding Framework for Dynamic Human Face Reconstruction — Reconstructing dynamic faces from fMRI requires combining global appearance context with explicit 3D geometry-aware motion control.
• Fast Whole-Brain, Geometry-Aware Functional Alignment for Cross-Subject Decoding — SpectralOT leverages cortical geometry via Laplace-Beltrami eigenmodes for fast, accurate, and tunable fMRI functional alignment.
• DVFace: Spatio-Temporal Dual-Prior Diffusion for Video Face Restoration — DVFace uses a one-step, dual-prior diffusion framework for efficient, high-quality video face restoration.
• FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision — Hybrid 2D/3D supervision with direct 3D expression parameters and intensity-aware emotion loss improves emotion-rich 3D face reconstruction.
• LBFTI: Layer-Based Facial Template Inversion for Identity-Preserving Fine-Grained Face Reconstruction — LBFTI reconstructs identity-preserving faces from templates by decomposing images into foreground, midground, and background layers.
• Realistic Face Reconstruction from Facial Embeddings via Diffusion Models — FEM reconstructs high-resolution faces from embeddings, exposing privacy risks in FR and PPFR systems.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.