Score Accuracy Along the Forward Diffusion Does Not Certify Numerical Stability in Diffusion Sampling

2026-07-09 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, quick

Summary

The paper "Score Accuracy Along the Forward Diffusion Does Not Certify Numerical Stability in Diffusion Sampling" demonstrates that small forward-marginal L^2 error in score matching does not guarantee numerical stability for discretized reverse-time samplers in diffusion models. Researchers constructed a smooth score field with arbitrarily small forward-marginal error, where the learned reverse-time process is nonexplosive and path-space close to the exact process. However, its Euler--Maruyama discretizations converge in probability but exhibit diverging positive moments, leading to diverging Wasserstein distances W_p for p ≥ 1. This issue can persist even within a fixed finite neural architecture. The study also presents a family of bounded, globally Lipschitz denoisers where both forward-marginal error and path-space total variation tend to zero, yet Euler--Maruyama endpoints diverge in every W_p. For compactly supported data, projecting the learned denoiser onto a known bounded closed convex set preserves pointwise accuracy, provides grid-uniform moment bounds, and ensures Wasserstein convergence under mild local regularity. Experiments with a small fixed DiT-style network confirmed large growth along rare numerical trajectories, which was suppressed by denoiser projection.

Key takeaway

For Machine Learning Engineers developing diffusion models, relying solely on forward-marginal score accuracy metrics is insufficient for ensuring numerical stability. You should implement denoiser projection, especially when working with compactly supported data, to prevent diverging moments and ensure robust Wasserstein convergence in your Euler--Maruyama discretizations. This approach helps suppress large growth along rare numerical trajectories, improving overall model reliability.

Key insights

Small forward-marginal score error does not guarantee numerical stability or Wasserstein convergence in diffusion model sampling.

Principles

• Forward-marginal error doesn't imply reverse-time stability.
• Weak convergence can coexist with diverging moments.
• Denoiser projection improves stability for bounded data.

Method

Projecting the learned denoiser onto a known bounded closed convex set containing data support preserves accuracy and yields Wasserstein convergence.

In practice

• Implement denoiser projection for diffusion models.
• Monitor moment divergence in Euler--Maruyama samplers.
• Evaluate W_p distances beyond weak convergence.

Topics

• Diffusion Models
• Numerical Stability
• Score Matching
• Euler--Maruyama Discretization
• Wasserstein Distance
• Denoiser Projection

Best for: Research Scientist, Computer Vision Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• Error Bounds for a Diffusion Model-Based Drift Estimator — Theoretical guarantees for a diffusion model-based drift estimator in SDEs are now established through an explicit risk bound.
• Wasserstein Convergence of ODE-Based Samplers in Decentralized Diffusion Model via Velocity Field Decomposition — The first W₂ convergence guarantee for decentralized diffusion models with ODE-based sampling is established, showing ℮(N⁻¹/²+ε) rate.
• LESS Is More: Mutual-Stability Sampling for Diffusion Language Models — Adaptive sampling based on token stability significantly enhances diffusion LLM inference efficiency.
• From Score Approximation to Distribution Approximation in Score-Based Diffusion Models — A rigorous theorem links neural network score function approximation directly to generated probability distribution approximation in diffusion models.
• A Stabilized Path-Space Approach to Diffusion-Based Posterior Sampling — A stabilized path-space framework improves diffusion posterior sampling by connecting it to stochastic optimal control and removing bias.
• Uncertainty Estimation for Molecular Diffusion Models — A post-hoc method estimates per-sample uncertainty in molecular diffusion models using Laplace approximation to improve sample quality.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.