Dual-Domain Equivariant Generative Adversarial Network for Multimodal CT-PET Synthesis

2026-06-11 · Source: Computer Vision and Pattern Recognition · Field: Science & Research — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Expert, quick

Summary

The Dual-Domain Equivariant Generative Adversarial Network (DDE-GAN) is a novel approach for multimodal CT-PET image synthesis. Unlike traditional GANs that operate only in the spatial domain and often lack geometric consistency, DDE-GAN learns jointly from both spatial and frequency (Fourier) domains to capture comprehensive anatomical and spectral information. It integrates rotational equivariance, derived from CT and PET measurement physics, into both generator and discriminator losses, ensuring consistent responses under rotations and improving anatomical accuracy. A hierarchical dual-domain training strategy further enforces intra- and inter-domain consistency through multi-stage loss functions. Evaluated on the HECKTOR 2022 CT-PET dataset, DDE-GAN achieved superior synthesis quality compared to baseline models, demonstrating enhanced accuracy and robustness for applications like PET completion and data augmentation.

Key takeaway

For Machine Learning Engineers developing medical imaging solutions, if you are struggling with structural fidelity in multimodal synthesis, consider integrating dual-domain learning and geometric equivariance. Your models will achieve superior accuracy and robustness, particularly for tasks like PET completion or data augmentation. This approach can significantly improve the quality of synthetic medical images, reducing artifacts and enhancing diagnostic utility.

Key insights

Dual-domain learning and rotational equivariance significantly enhance multimodal medical image synthesis accuracy and robustness.

Principles

• Jointly learn spatial and frequency domains.
• Embed rotational equivariance for consistency.
• Use hierarchical multi-stage loss functions.

Method

DDE-GAN employs a hierarchical dual-domain training strategy, integrating spatial and Fourier domain learning with rotational equivariance in generator and discriminator losses to enforce intra- and inter-domain consistency.

In practice

• Apply DDE-GAN for PET completion.
• Use DDE-GAN for medical data augmentation.

Topics

• CT-PET Synthesis
• Generative Adversarial Networks
• Rotational Equivariance
• Dual-Domain Learning
• Medical Image Processing
• HECKTOR 2022 Dataset

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Dual-Domain Equivariant Generative Adversarial Network for Multimodal CT-PET Synthesis — Combining dual-domain learning with geometric equivariance significantly enhances multimodal medical image synthesis accuracy and robustness.
• Contrast-Informed Augmentation and Domain-Adversarial Training for Adult-to-Neonatal MR Reconstruction Generalization — Contrast-informed augmentation and domain-adversarial training enhance deep learning MR reconstruction generalization from adult to neonatal data.
• Unified MRI Brain Image Translation via Hierarchical Tumor Structure Comparison — HTSCGAN improves multi-modal MRI brain image translation by integrating hierarchical tumor structural information using a GAN with specific modules and loss functions.
• Virtual Scanning for NSCLC Histology: Investigating the Discriminatory Power of Synthetic PET — Synthetic PET data can significantly enhance NSCLC histological classification when combined with CT scans.
• Cross-Source Supervision for Bone Infection Segmentation in Dual-Modality PET-CT — Decoupled dual-source learning with PET-CT fusion improves bone infection segmentation by accommodating diverse clinical annotation philosophies.
• Generative Deep Learning for Computational Destaining and Restaining of Unregistered Digital Pathology Images — Preprocessing-based domain adaptation enables cGAN models to generalize for computational pathology across institutions without retraining.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.