A Dual Path Framework with Hotspot Guided Fusion for Three Dimensional CT to PET Synthesis in Head and Neck Cancer

2026-07-27 · Source: cs.CV updates on arXiv.org · Field: Health & Wellbeing — Health & Medical Research, Medical Devices & Health Technology · Depth: Expert, extended

Summary

A three-dimensional dual-path deep learning framework is introduced for synthesizing PET-like images directly from routine CT scans in head and neck cancer patients. Evaluated on 44 patients from the publicly available QIN-HEADNECK dataset using five-fold cross-validation, the framework integrates a regression U-Net for quantitative SUV estimation and a conditional generative adversarial network (cGAN) for realistic PET texture. Their outputs are fused using hotspot-guided Laplacian pyramid blending. The framework achieved a mean absolute error of 0.00395, PSNR of 39.19 dB, and SSIM of 0.9634 on reconstructed 3D PET volumes. It accurately localized many FDG-avid lesions and produced anatomically realistic background texture, though it systematically underestimated SUV within highly metabolically active tumor regions, a limitation consistent with previous studies. This model serves as a proof-of-concept for estimating metabolically informative imaging from CT, not as a diagnostic replacement.

Key takeaway

For AI Scientists and Machine Learning Engineers developing medical image synthesis models, you should consider adopting a dual-path optimization strategy. This approach allows you to independently refine quantitative accuracy and perceptual realism, then selectively combine them using anatomically informed fusion. This can mitigate the common trade-off between smooth, accurate quantitative outputs and sharp, realistic textures, providing a more robust framework for generating clinically useful synthetic images, especially where precise lesion quantification is critical.

Key insights

Separating quantitative and perceptual objectives in medical image synthesis improves both accuracy and realism.

Principles

• Optimize quantitative SUV estimation and perceptual realism independently.
• Fuse predictions using hotspot-guided blending for region-specific strengths.
• Regression U-Nets provide stable SUV; cGANs yield realistic texture.

Method

A dual-path architecture uses a regression U-Net for voxel-wise SUV and a cGAN for realistic PET texture. Outputs are fused via hotspot-guided 3D Laplacian pyramid blending, prioritizing regression in active regions.

In practice

• Use a 3D U-Net with multi-head self-attention for regression.
• Employ a 3D PatchGAN discriminator for cGAN realism.
• Generate hotspot masks from regression predictions (SUV ≈ 2.5 threshold).

Topics

• CT-to-PET Synthesis
• Head and Neck Cancer
• Deep Learning
• Generative Adversarial Networks
• U-Net
• Laplacian Pyramid Blending
• Medical Imaging

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Dual-Domain Equivariant Generative Adversarial Network for Multimodal CT-PET Synthesis — Dual-domain learning and rotational equivariance significantly enhance multimodal medical image synthesis accuracy and robustness.
• WING: A Window-Prior-Based Generative Network with Gated Inception for Cross-Modality CT Synthesis — CT synthesis benefits from window-prior-based regression to manage intensity distributions and improve structural fidelity.
• Generative Deep Learning for Computational Destaining and Restaining of Unregistered Digital Pathology Images — Preprocessing-based domain adaptation enables cGAN models to generalize for computational pathology across institutions without retraining.
• Local Label-Informed Feature Transfer for Generating Ground-Truth Medical Images: A Comparison of GAN- and Diffusion-Based Approaches — The LLIFT framework generates realistic, semi-synthetic medical images with controlled lesions, bypassing pixel-level annotation needs.
• Virtual Scanning for NSCLC Histology: Investigating the Discriminatory Power of Synthetic PET — Synthetic PET data can significantly enhance NSCLC histological classification when combined with CT scans.
• Anatomy-Guided Residual Motion Diffusion for Controllable 4D Cardiac MRI Synthesis — A cascaded latent diffusion model synthesizes controllable, anatomically consistent 4D cardiac MRI, improving AI model robustness with limited data.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.