Stitching and dimensionality effects on large artificially generated volume datasets

2026-06-18 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition · Depth: Expert, quick

Summary

This study investigated stitching artifacts and dimensionality effects (2D vs 3D) in generating large images via deep learning, specifically using cycleGAN models trained on cryo-electron microscopy datasets. The research addressed how patching input data to accommodate hardware memory limitations and then assembling output patches can introduce artifacts. It evaluated both perceptual quality and performance on downstream mitochondria segmentation. Key findings include that FID scores fail to detect subtle stitching artifacts that significantly impact segmentation. While 3D models with artifact-free stitching marginally outperform 2D models on downstream tasks, the improvement barely justifies the computational cost. Additionally, 2D models train more stably due to larger batch sizes. The study also demonstrated that ensembling predictions from three orthogonal directions can improve low-quality volumes but offers no benefit for high-quality outputs, highlighting the need for careful artifact mitigation and the inadequacy of perceptual metrics for evaluating domain adaptation in biomedical imaging.

Key takeaway

For Machine Learning Engineers developing generative models for large scientific datasets, particularly in biomedical imaging, you must prioritize robust stitching artifact mitigation. Relying solely on perceptual metrics like FID scores is insufficient, as subtle artifacts can severely impact downstream segmentation performance. Consider 2D models for more stable training and lower computational overhead, reserving 3D models only when marginal performance gains definitively justify the increased cost. Additionally, explore ensembling predictions for improving low-quality volume outputs.

Key insights

Stitching artifacts in large image generation significantly degrade downstream segmentation, undetected by perceptual metrics like FID scores.

Principles

• Perceptual metrics alone are insufficient for biomedical image quality evaluation.
• 3D models offer marginal gains over 2D for high computational cost.
• 2D models provide more stable training with larger batch sizes.

Method

Investigated three stitching approaches and two patch dimensionalities (2D vs 3D) using cycleGAN models on cryo-electron microscopy datasets, evaluating perceptual quality and downstream segmentation.

In practice

• Mitigate stitching artifacts for generative model performance.
• Consider 2D models for stable training and lower cost.
• Ensemble predictions for low-quality volume improvement.

Topics

• Generative Models
• Stitching Artifacts
• CycleGAN
• Cryo-electron Microscopy
• Image Segmentation
• Perceptual Metrics

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation — FID scores are highly variable due to training randomness, making single reported numbers unreliable and requiring new evaluation protocols.
• Aligning Latent Geometry for Spherical Flow Matching in Image Generation — Spherical Flow Matching improves image generation by aligning latent geometry on a sphere using geodesic paths.
• Calibrating Generative Models to Feature Distributions with MMD Finetuning — kCGM corrects generative model miscalibration by matching feature distributions with MMD and KL regularization, improving validity.
• The Sequence Knowledge # 780: Synthetic Data for Image Models — Synthetic data generation is crucial for robust vision systems, overcoming real-world data limitations through controlled pixel creation.
• STiTch: Semantic Transition and Transportation in Collaboration for Training-Free Zero-Shot Composed Image Retrieval — Training-free zero-shot CIR improves by refining LLM-generated captions and using set-to-set alignment for diverse compositions.
• Data Synthesis Improves 3D Myotube Instance Segmentation — Biophysics-driven data synthesis enables effective instance segmentation in annotation-scarce biomedical domains.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.