Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion

2026-07-14 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

Sticky Jump Diffusions (SJDs) are introduced as continuous-time Markov processes operating on ℝᵈ, utilizing discrete token embeddings as anchors. In the forward process, anchors release their mass at a specified hazard rate, which then diffuses in a continuous ambient space. The time-reversed process couples a score-driven Stochastic Differential Equation (SDE) with a sticky jump kernel, whose rate and destination are determined by flux balance with the forward law. Learning this process involves Denoising Hazard Matching (DHM), an analogue of denoising score matching, using a single denoising classifier trained via simulation-free cross-entropy. SJDs unify existing masked, continuous, and hybrid diffusion models as specific limits, providing a principled explanation for their inherent features. Furthermore, SJDs introduce a new design space through the unsticking kernel, enabling cross-position blending that improves performance on tasks like CIFAR-10 (achieving 1.3 FID points below the identity-kernel hybrid), Text8, and Sudoku.

Key takeaway

For AI Engineers developing generative models for discrete data, Sticky Jump Diffusions offer a unified, principled framework. You should consider implementing SJDs, especially leveraging the cross-position blending kernel, to improve performance on tasks with inherent dependency structures like images, text, or combinatorial puzzles. This approach simplifies reverse sampler design by deriving commitment rates and destinations directly from forward dynamics, potentially reducing hand-tuning and enhancing model robustness.

Key insights

Sticky Jump Diffusions unify diverse diffusion models by deriving reverse dynamics from flux balance and a single classifier.

Principles

• Reverse dynamics are fixed by forward flux balance.
• A single classifier estimates both score and reverse hazards.
• Unsticking kernel is a new design space for inductive bias.

Method

Sticky Jump Diffusions are learned via Denoising Hazard Matching (DHM). This involves training a single denoising classifier with simulation-free cross-entropy to estimate both the score and per-anchor reverse intensities through analytic reweighting.

In practice

• Implement cross-position blending for structured data.
• Use SJD for improved image, text, and Sudoku generation.
• Leverage flux balance for principled reverse sampling.

Topics

• Sticky Jump Diffusions
• Diffusion Models
• Generative Modeling
• Denoising Hazard Matching
• Markov Processes
• Discrete Data Generation
• Inductive Bias

Code references

• PascalJD/sticky-jump-diffusions

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• A Unified Approach to Analysis and Design of Denoising Markov Models — A unified mathematical framework rigorously designs and analyzes denoising Markov models, integrating diverse dynamics and objectives.
• Wasserstein Convergence of ODE-Based Samplers in Decentralized Diffusion Model via Velocity Field Decomposition — The first W₂ convergence guarantee for decentralized diffusion models with ODE-based sampling is established, showing ℮(N⁻¹/²+ε) rate.
• Statistical guarantees for denoising reflected diffusion models — Reflected diffusion models provide statistical guarantees for generative AI, resolving unbounded state space issues in implementations.
• Towards a Unified Generative Model for Scarce Time Series with Domain Experts — TimeMoDE unifies Diffusion Transformers and Mixture-of-Experts for robust time series generation in data-scarce settings.
• A Mathematical Introduction to Diffusion Models — This introduction mathematically grounds diffusion models by tracing sampling dynamics from classical methods to modern samplers.
• Initialization-Aware Score-Based Diffusion Sampling — Optimizing the backward process initialization in SGMs significantly reduces computational cost while maintaining generative quality.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.