Spectral Prior for Reducing Exposure Bias in Diffusion Models

2026-07-27 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

Diffusion models often suffer from exposure bias, where iterative sampling errors accumulate. This work identifies systematic frequency-dependent discrepancies, or SNR errors, between training and inference, noting that their direction varies across models and timesteps. Spectral Alignment (SPA) is proposed as a lightweight, guidance-based method to correct this. SPA calibrates intermediate prediction power spectra to a pre-computed prior through offline fitting of a parametric spectrum model and inference-time FFT-based gradient guidance. It introduces minimal computational overhead (3-4%) and consistently improves image quality across diverse architectures, including pixel-space models (DDPM, ADM), latent diffusion models (SD2.0, SDXL), and flow-matching models (SD3.5, FLUX), outperforming other noise correction baselines.

Key takeaway

For Machine Learning Engineers deploying diffusion models, you should consider integrating Spectral Alignment (SPA) to mitigate exposure bias and enhance image quality. SPA offers consistent improvements across various model architectures, including SDXL and FLUX, with only 3-4% computational overhead. This allows you to refine generated images, particularly improving object shapes and details, without extensive retraining or complex modifications to existing pipelines.

Key insights

Diffusion models' exposure bias stems from frequency-dependent SNR errors, correctable via spectral alignment guidance.

Principles

• Frequency-dependent SNR errors cause exposure bias.
• Fixed correction rules do not generalize across models.
• Data-driven spectral alignment improves image quality.

Method

SPA involves offline parametric spectrum model fitting from training data, followed by inference-time guidance using efficient FFT-based gradient computation to steer predictions toward the target spectrum.

In practice

• Apply SPA to existing pretrained diffusion models.
• Refine flawed object shapes and enhance details.
• Improve lower-quality generated samples.

Topics

• Diffusion Models
• Exposure Bias
• Spectral Alignment
• Image Generation
• Latent Diffusion Models
• FFT Guidance

Code references

• SonyResearch/SPA

Best for: Research Scientist, Computer Vision Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• Spectral Prior for Reducing Exposure Bias in Diffusion Models — Diffusion models' exposure bias stems from frequency-dependent SNR errors, addressable by spectral alignment guidance.
• Spectral Diffusion Models on the Sphere — Spherical data diffusion models require spectral domain SDEs with non-isotropic noise due to geometric constraints.
• DVFace: Spatio-Temporal Dual-Prior Diffusion for Video Face Restoration — DVFace uses a one-step, dual-prior diffusion framework for efficient, high-quality video face restoration.
• Not All Patches are Equal: Sampling Matters for Visible-Infrared Pre-Training — Not all VIS-IR patches are equally alignable, and weighting them by importance improves representation learning.
• Rethinking Token Reduction for Diffusion Models via Output-Similarity-Awareness — DiTo improves Diffusion Transformer efficiency by focusing token reduction on output similarity, not just input similarity.
• MTVDiff: Multimodal Conditional Latent Diffusion for Enhanced Thermal-to-Visible Face Translation — Multimodal latent diffusion, integrating thermal, depth, and text, significantly enhances thermal-to-visible face translation and identity preservation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.