The Within-Orbit Adaptive Leapfrog No-U-Turn Sampler

2025-12-31 · Source: JMLR · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

The Within-Orbit Adaptive Leapfrog No-U-Turn Sampler (WALNUTS) is a new Markov chain Monte Carlo (MCMC) method introduced by Nawaf Bou-Rabee, Bob Carpenter, Tore Selland Kleppe, and Sifan Liu in 2026. This sampler generalizes the No-U-Turn Sampler (NUTS) to address the challenge of locally adapting the leapfrog integrator's step size dynamically, which is crucial for posterior distributions with multiscale geometries. WALNUTS adapts the step size at fixed intervals of simulated time, selecting the largest step size from a dyadic schedule that maintains energy error below a user-specified threshold. Like NUTS, it uses biased progressive state selection to favor states further from the initial point. Empirical evaluations on multiscale target distributions, including Neal's funnel and the Stock-Watson stochastic volatility time-series model, demonstrate that WALNUTS significantly improves sampling efficiency and robustness compared to NUTS.

Key takeaway

For data scientists and machine learning engineers working with complex posterior distributions, especially those exhibiting multiscale geometries, WALNUTS offers a significant advancement. You should consider integrating WALNUTS into your MCMC workflows to achieve substantial improvements in sampling efficiency and robustness compared to traditional NUTS. This can accelerate model convergence and provide more reliable inference for challenging statistical models.

Key insights

WALNUTS dynamically adapts leapfrog step sizes within NUTS orbits, improving efficiency for multiscale posteriors.

Principles

• Local adaptation of step size is crucial for multiscale posteriors.
• Reversibility preservation is difficult with local adaptation.
• Biased state selection favors distant points in orbit.

Method

WALNUTS adapts leapfrog step size at fixed intervals, selecting the largest from a dyadic schedule that keeps energy error below a threshold. It uses biased progressive state selection.

In practice

• Apply WALNUTS to multiscale target distributions.
• Use for models like Neal's funnel.
• Test with stochastic volatility time-series models.

Topics

• Markov chain Monte Carlo
• No-U-Turn Sampler
• Hamiltonian Monte Carlo
• Leapfrog Integrator
• Multiscale Posterior Distributions
• Sampling Efficiency

Code references

• JmlrOrg/v27

Best for: Research Scientist, AI Scientist, Data Scientist, Machine Learning Engineer

Related on AIssential

• Nutpie: state-of-the-art mass matrix adaptation for HMC — The new "Nutpie sampler" significantly outperforms Stan's and PyMC's samplers, typically being twice as fast, primarily due to superior mass-matrix adaptation within the Hamiltonian Monte Carlo (HMC) framework.
• Task-Error Residual Learning for Real-Robot Five-Ball Juggling — Directional task error and an informative prior are crucial for efficient, stable real-robot residual learning.
• Entropic Mirror Monte Carlo — A novel adaptive Monte Carlo scheme improves importance sampling efficiency for complex distributions via global sampling and delayed weighting.
• Large-scale empirical tuning and comparison of default optimizers for variational inference — A selection of 5 adaptive stochastic optimizers reliably approximates optimal performance for black-box variational inference without extensive tuning.
• Time-Aware Latent Space Bayesian Optimization — TALBO adapts latent-space Bayesian optimization to dynamic objectives by integrating time into both the surrogate and generative representation.
• Tempered Sequential Monte Carlo for Trajectory and Policy Optimization with Differentiable Dynamics — TSMC optimizes trajectories and policies by framing controller design as inference, using tempered sampling and gradient-based rejuvenation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by JMLR.