Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation

2026-06-18 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

The Pixel-Level Residual Diffusion Transformer (PRDiT) is a scalable generative framework designed to synthesize high-quality 3D medical volumes directly at the voxel-level. Addressing the computational and optimization challenges of existing models for high-resolution 3D CT generation, PRDiT employs a two-stage training architecture. This includes a local denoiser, an MLP-based blind estimator operating on overlapping 3D patches to efficiently separate low-frequency structures, and a global residual diffusion transformer utilizing memory-efficient attention to model and refine high-frequency residuals across entire volumes. This coarse-to-fine modeling strategy simplifies optimization, enhances training stability, and effectively preserves subtle structures without an autoencoder bottleneck. Experiments on the LIDC-IDRI and RAD-ChestCT datasets show PRDiT consistently outperforms HA-GAN, 3D LDM, and WDM-3D, achieving significantly lower 3D FID, MMD, and Wasserstein distance scores.

Key takeaway

For AI Scientists or Machine Learning Engineers developing generative models for 3D medical imaging, PRDiT presents a robust alternative. Its two-stage architecture, separating low and high-frequency details, significantly improves the quality and stability of high-resolution 3D CT volume generation. You should evaluate PRDiT's coarse-to-fine modeling strategy to overcome computational demands and enhance fine detail preservation in your applications, potentially achieving superior results compared to current state-of-the-art models.

Key insights

PRDiT uses a two-stage, coarse-to-fine diffusion transformer to generate high-resolution 3D medical volumes directly at voxel-level.

Principles

• Coarse-to-fine modeling simplifies optimization.
• Separating frequency components preserves subtle details.
• Memory-efficient attention scales to entire volumes.

Method

A local MLP-based denoiser processes 3D patches for low-frequency structures, followed by a global residual diffusion transformer refining high-frequency residuals.

In practice

• Synthesize high-quality 3D CT volumes.
• Enhance detail preservation in medical imaging.

Topics

• 3D CT Volume Generation
• Diffusion Transformers
• Medical Imaging
• Voxel-level Synthesis
• Generative AI
• LIDC-IDRI Dataset

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• RiT: Vanilla Diffusion Transformers Suffice in Representation Space — DINOv2's favorable geometric properties enable vanilla Diffusion Transformers with "x"-prediction to achieve superior, efficient image generation.
• Modality Forcing for Scalable Spatial Generation — Modality Forcing enables scalable, joint image-depth generation from sparse data by employing T2I priors with a simple post-training recipe.
• MMDiff: Extending Diffusion Transformers for Multi-Modal Generation — Perceptual information in diffusion transformers is temporally distributed, enabling multi-modal generation via lightweight decoders on frozen backbones.
• Conditional Latent Diffusion Model with Fourier-based Motion Modelling for Virtual Population Synthesis — A conditional latent diffusion model uses Fourier-based motion modeling to synthesize cycle-consistent 3D+t cardiac meshes for in-silico trials.
• GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos — GenLCA generates photorealistic 3D avatars from partial 2D video data using a novel visibility-aware diffusion training strategy.
• PPDM: Pixel Puzzling Diffusion Model for Speed and Memory Efficient Volumetric Medical Image Translation — PPDM efficiently translates 3D medical images by trading spatial resolution for channel depth and focusing on task-relevant residuals.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.