Synthesize Realistic 3D Medical Images at Scale to Ship Pre‑Trained Models

2026-05-22 · Source: NVIDIA Technical Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Medical Devices & Health Technology · Depth: Advanced, medium

Summary

NVIDIA introduced Medical AI for Synthetic Imaging (MAISI) in 2024, a generative model for synthesizing high-resolution 3D CT volumes with pixel-level anatomical segmentation. Building on this, NV-Generate-CTMR, including MAISI-v2 with Latent Rectified Flow, provides an open-source framework for synthetic CT and MRI generation, offering 33x acceleration in inference speed. A new model, NV-Generate-MR-Brain, extends this for human brain anatomy and structure segmentation, trained on the MR-RATE dataset. MR-RATE, the world's largest open-source multimodal MRI dataset, comprises 100,000 brain MRI studies from over 83,000 patients, totaling about 700,000 volumes, each paired with de-identified radiology reports and metadata. This framework addresses data scarcity and privacy in medical AI by enabling scalable generation of realistic 3D volumes and paired segmentations, supporting diverse clinical scenarios and applications like anomaly detection and cancer classification.

Key takeaway

For AI Scientists and Machine Learning Engineers developing medical imaging solutions, NVIDIA's NV-Generate-CTMR and NV-Generate-MR-Brain offer a critical resource. You can now generate high-resolution, anatomically consistent 3D medical images and segmentations at scale, directly addressing data scarcity and privacy concerns. Integrate these open-source models into your pipelines to augment datasets, simulate rare pathologies, and accelerate robust medical AI system development. This approach significantly lowers technical and compute barriers, allowing you to focus on model generalization and clinical impact.

Key insights

NVIDIA's MAISI and NV-Generate frameworks provide scalable, open-source 3D medical image synthesis to overcome data bottlenecks.

Principles

• Synthetic data augments limited training sets.
• Open-source models reduce development barriers.
• Fine-tuning is faster and more efficient than training from scratch.

Method

NV-Generate-CTMR uses MAISI-v1 (Latent DDPM) for diversity or MAISI-v2 (Latent Rectified Flow) for 33x faster, higher-quality 3D medical image synthesis, supporting flexible voxel and volume sizes.

In practice

• Generate synthetic CT/MRI volumes with segmentations.
• Fine-tune models for new anatomies or modalities.
• Use generated data for anomaly detection or cancer classification.

Topics

• Medical Imaging AI
• Synthetic Data Generation
• 3D CT/MRI Synthesis
• NVIDIA MAISI
• MR-RATE Dataset
• Latent Rectified Flow

Code references

• NVIDIA-Medtech/NV-Generate-CTMR

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Anatomy-Guided Residual Motion Diffusion for Controllable 4D Cardiac MRI Synthesis — A cascaded latent diffusion model synthesizes controllable, anatomically consistent 4D cardiac MRI, improving AI model robustness with limited data.
• Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers — A 1.3B-parameter generative foundation model for chest radiographs achieves expert-indistinguishable synthesis and controllable editing.
• Impact of Synthetic Lesional MR Images in Automated Focal Cortical Dysplasia Detection in Low-Data Scenarios — Synthetic FCD-MRIs can augment low-data scenarios, reducing labeled data needs by ~20% while maintaining detection sensitivity.
• Learning neuroimaging models from health system-scale data — Health system-scale data enables AI foundation models to achieve superior, fair, and explainable neuroimaging diagnostics.
• BrainG3N: A Dual-Purpose Tokenizer for Controllable 3D Brain MRI Generation — A novel dual-purpose tokenizer enables a unified 3D brain-MRI embedding space for clinical tasks and controllable generation.
• From Baseline to Follow-Up: Counterfactual Spine DXA Image Synthesis in UK Biobank Using a Causal Hierarchical Variational Autoencoder — The CHVAE model synthesizes anatomically plausible, causally consistent spine DXA images by intervening on age, addressing anatomical variation challenges.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Technical Blog.