Guidance Breaks the Fitted Operator: A Terminal-Fitted Repair for Classifier-Free Guidance

2026-07-08 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, quick

Summary

Classifier-free guidance (CFG), a standard method for strengthening class-conditioning in diffusion and flow-matching samplers, exhibits oversaturation and instability at high guidance levels. An analysis reveals that guidance re-stiffens the discriminative subspace to an anomalous exponent 1+w, rendering the deterministic DDIM step unfitted and causing its guided residual to diverge as sigma_min approaches zero. This leads to a guided clock barrier and one-step oversaturation, identified as a solver artifact. The research proposes a one-coefficient, zero-extra-NFE repair: replacing CFG's w(r-1) with r^(1+w)-r on the guidance direction. This modification eliminates CFG's sigma_min-divergent blow-up and achieves first-order accuracy against the exact guided flow. Evaluated on CIFAR-10 checkpoints and Stable Diffusion 1.5 DDIM, the repair stabilizes high-guidance scenarios, reducing residual amplification and saturation, securing 9/9 point-FID wins, and maintaining classifier-proxy target accuracy, though it is not a universal image-quality improvement.

Key takeaway

For Machine Learning Engineers optimizing diffusion models with classifier-free guidance, if you encounter oversaturation or instability at high guidance levels, consider implementing the proposed repair. By replacing CFG's w(r-1) with r^(1+w)-r in the guidance direction, you can stabilize your models and achieve better FID scores without increasing inference steps. This modification addresses a core numerical issue, offering a targeted solution for high-guidance scenarios.

Key insights

CFG's high-guidance instability stems from a numerical solver artifact, repairable with a simple, zero-cost modification.

Principles

• Guidance re-stiffens discriminative subspaces to exponent 1+w.
• DDIM's fitted operator property breaks under high guidance.
• Oversaturation can be a solver artifact, not a continuous law issue.

Method

Replace CFG's w(r-1) with r^(1+w)-r on the guidance direction to repair the fitted operator, removing sigma_min-divergent blow-up.

In practice

• Apply r^(1+w)-r substitution for high-guidance CFG.
• Stabilize diffusion models without extra inference steps.
• Improve FID scores in specific high-guidance contexts.

Topics

• Classifier-Free Guidance
• Diffusion Models
• Flow Matching Samplers
• Numerical Analysis
• DDIM
• Image Generation
• Model Stabilization

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• CFG-Ctrl: Control-Based Classifier-Free Diffusion Guidance — CFG can be reinterpreted as a control system, improving stability and semantic alignment with nonlinear methods.
• Conflict-Aware Additive Guidance for Flow Models under Compositional Rewards — Gradient misalignment causes off-manifold drift when composing multiple constraints in guided sampling for flow models.
• Distilling Drifting Transformers with Representation Autoencoders — Drift-RAE enables stable, high-performance distillation of flow models in semantically rich Representation Autoencoder latent spaces.
• Learning to Solve Generative ODEs Beyond the Linear Span — Generative ODE solvers can overcome span limitations by learning a spatial residual operator.
• Reward Weighted Classifier-Free Guidance as Policy Improvement in Autoregressive Models — RCFG enables test-time optimization of autoregressive models for arbitrary reward functions without retraining.
• Don't Settle at the Mode! Mitigating Diversity Collapse in Pretrained Flow Models via Feature Self-Guidance — Feature Self-Guidance efficiently mitigates diversity collapse in flow models by dispersing internal features and regularizing them to the data manifold.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.