Dual-Domain Equivariant Generative Adversarial Network for Multimodal CT-PET Synthesis

2026-06-12 · Source: cs.AI updates on arXiv.org · Field: Science & Research — Artificial Intelligence & Machine Learning, Health & Medical Research, Mathematics & Computational Sciences · Depth: Expert, long

Summary

The Dual-Domain Equivariant Generative Adversarial Network (DDE-GAN) is introduced for multimodal CT-PET image synthesis, addressing limitations of traditional GANs that often lack geometric consistency and operate only in the spatial domain. DDE-GAN jointly learns from both spatial and frequency (Fourier) domains, capturing complementary anatomical and spectral information. It integrates rotational equivariance, inherent in CT and PET physics, into the loss functions of both the generator and discriminator to ensure consistent responses under rotations and improve anatomical accuracy. A hierarchical dual-domain training strategy enforces intra- and inter-domain consistency through multi-stage loss functions. Evaluated on the HECKTOR 2022 CT-PET dataset, DDE-GAN achieved superior synthesis quality, demonstrating an approximate 6 dB increase in PSNR over baseline models and a 2.7 dB PSNR and 0.12 SSIM improvement compared to DD-GAN without equivariance. This method enhances multimodal image synthesis accuracy and robustness, enabling practical applications in PET completion and data augmentation.

Key takeaway

For machine learning engineers developing medical image synthesis models, you should consider integrating dual-domain learning and geometric equivariance. This approach, exemplified by DDE-GAN, significantly improves anatomical accuracy and robustness in CT-PET synthesis, reducing artifacts and enhancing realism. You can apply this to generate high-quality PET images from CT or augment your multimodal training datasets, leading to more reliable clinical applications.

Key insights

Combining dual-domain learning with geometric equivariance significantly enhances multimodal medical image synthesis accuracy and robustness.

Principles

• Jointly process image data in spatial and frequency domains for comprehensive feature capture.
• Embed rotational equivariance into GAN loss to preserve geometric consistency.
• Employ hierarchical multi-stage training for intra- and inter-domain consistency.

Method

Train networks in three stages: S1 for intra-domain consistency (L1 norm), S2 for inter-domain consistency (comparing generated images across domains), and S3 for enforcing rotational equivariance via a specific loss function.

In practice

• Synthesize PET images from low-dose CT scans.
• Augment multimodal training datasets with geometrically consistent images.

Topics

• CT-PET Synthesis
• Generative Adversarial Networks
• Dual-Domain Learning
• Rotational Equivariance
• Medical Image Synthesis
• Data Augmentation

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Dual-Domain Equivariant Generative Adversarial Network for Multimodal CT-PET Synthesis — Dual-domain learning and rotational equivariance significantly enhance multimodal medical image synthesis accuracy and robustness.
• An Improved Generative Adversarial Network for Micro-Resistivity Imaging Logging Restoration — A specialized GAN architecture significantly improves micro-resistivity imaging log restoration by integrating multi-scale feature learning and dual discrimination.
• Lecture 6: Gauge-equivariant Mesh CNN - Pim de Haan — Gauge-equivariant Mesh CNNs enable anisotropic convolutions on discrete manifolds by handling local orientation ambiguities.
• Realistic noise synthesis reduces bias and improves tissue microstructure estimation with supervised machine learning — Realistic noise synthesis in simulated data is crucial for unbiased dMRI microstructure estimation, especially at low SNR.
• IEEE Transactions on Neural Networks and Learning Systems, Volume 37, Issue 3, March 2026 — The issue highlights advancements across neural networks, computer vision, and machine learning applications.
• EqGINO: Equivariant Geometry-Informed Fourier Neural Operators for 3D PDEs — EqGINO combines FNOs with spectral isotropy to achieve 3D PDE equivariance and robust generalization.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.