A unified perspective on fine-tuning and sampling with diffusion and flow models

2026-05-04 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

This research unifies various approaches for training diffusion and flow generative models to sample from target distributions, specifically those defined by an exponential tilting of a base density. This framework addresses both sampling from unnormalized densities and reward fine-tuning of pre-trained models, integrating perspectives from stochastic optimal control (SOC) and non-equilibrium thermodynamics. The study identifies that Adjoint Matching/Sampling and Novel Score Matching methods exhibit finite gradient variance, unlike Target and Conditional Score Matching. It also establishes norm bounds for the lean adjoint ODE, providing theoretical backing for adjoint-based methods. Furthermore, the work adapts CMCD and NETS loss functions and introduces novel Crooks and Jarzynski identities for the exponential tilting context, validating these findings through reward fine-tuning experiments on Stable Diffusion 1.5 and 3.

Key takeaway

For research scientists working with generative models and fine-tuning, this unified framework offers critical insights into gradient variance and theoretical underpinnings for adjoint-based methods. You should consider the identified variance properties of different score matching techniques when designing or optimizing your training pipelines. This work provides a robust theoretical foundation for improving reward fine-tuning and sampling efficiency in diffusion models.

Key insights

A unified framework clarifies fine-tuning and sampling in diffusion/flow models via exponential tilting.

Principles

• Adjoint Matching has finite gradient variance.
• Lean adjoint ODEs are theoretically effective.

Method

The method unifies SOC and non-equilibrium thermodynamics, adapting CMCD/NETS loss functions and introducing Crooks/Jarzynski identities for exponential tilting.

In practice

• Fine-tune Stable Diffusion 1.5 and 3.
• Sample from unnormalized densities.

Topics

• Diffusion Models
• Flow Models
• Generative Models
• Exponential Tilting
• Stochastic Optimal Control

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Dimension-Free Convergence of Discrete Diffusion Models: Adjoint Equations Induce the Right Space — The adjoint-equation-based framework provides dimension-free convergence guarantees for discrete diffusion models, overcoming prior limitations with singular priors and state space size.
• A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decoding — A2D2 enables reward-guided fine-tuning for any-length discrete diffusion models via joint policy optimization and a novel loss, improving generation accuracy.
• Flow Matching for text2image models — Integrating a learned velocity field can deterministically transform noise into data.
• Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space — Language generation can be reframed as a stochastic optimal control problem to address model limitations.
• Variance Reduction for Expectations with Diffusion Teachers — CARV reduces Monte Carlo variance in frozen diffusion teacher gradients by hierarchically amortizing expensive computations.
• Conflict-Aware Additive Guidance for Flow Models under Compositional Rewards — Gradient misalignment causes off-manifold drift when composing multiple constraints in guided sampling for flow models.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.