Tensor Train Diffusion: Leveraging Low-Rank Structures for High-Dimensional Score-Based Sampling

2026-07-09 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

Tensor Train Diffusion (TTD) is a novel and efficient method for sampling from complex, unnormalized probability densities by solving high-dimensional Hamilton-Jacobi-Bellman (HJB) equations. This approach utilizes functional tensor train (FTT) representations, which exploit latent low-rank structures to efficiently approximate high-dimensional functions, enabling both model compression and rapid computation. TTD integrates this FTT representation with a backward-in-time iterative scheme derived from backward stochastic differential equations (BSDEs), offering a fast, robust, and accurate alternative to traditional neural network-based PDE solvers like PINNs, which often suffer from long training times and hyperparameter sensitivity. Numerical experiments demonstrate TTD's superior performance, outperforming existing diffusion-based samplers such as DIS and PIS in both speed and accuracy on challenging Multiwell problems (up to d=50 with 32 modes) and \u03c6^4 scalar field theory. Training times ranged from 1 minute for d=10 to 300 minutes for d=50.

Key takeaway

For research scientists developing generative models or performing Bayesian inference with complex, high-dimensional probability distributions, you should evaluate Tensor Train Diffusion (TTD). This method offers a faster, more accurate, and robust sampling solution than neural network-based PDE solvers. TTD reduces training times and hyperparameter sensitivity, enabling high-fidelity samples from challenging multimodal targets. Consider integrating TTD to accelerate your research and improve the quality of your generative processes.

Key insights

Low-rank tensor train approximations efficiently solve high-dimensional HJB equations for robust, fast, and accurate sampling from complex densities.

Principles

• High-dimensional functions often exhibit latent low-rank structures.
• HJB equations can be solved via backward SDEs.
• Tensor train formats enable efficient regression-based PDE solvers.

Method

TTD combines FTT representations with a backward-in-time iterative scheme derived from BSDEs, optimizing via alternating least squares (ALS) and adaptively adjusting regularization, rank, and basis degrees.

In practice

• Apply TTD to high-dimensional, multimodal sampling.
• Use Legendre, B-spline, or Fourier bases.
• Employ adaptive rank and basis degree selection.

Topics

• Tensor Train Diffusion
• HJB Equations
• High-Dimensional Sampling
• Low-Rank Approximation
• Diffusion Models
• BSDEs

Code references

• robertgruhlke/TTD

Best for: AI Scientist, Research Scientist

Related on AIssential

• Tensor Train Diffusion: Leveraging Low-Rank Structures for High-Dimensional Score-Based Sampling — Tensor Train Diffusion efficiently samples high-dimensional distributions by solving HJB PDEs using low-rank functional tensor train approximations.
• More Than Image Generators: A Science of Problem-Solving using Probability | Diffusion Models — Diffusion models combine deep learning with Langevin sampling to generate images by iteratively following noisy gradients of an image's probability distribution.
• Guaranteed Nonconvex Low-Rank Tensor Estimation via Scaled Gradient Descent — ScaledGD offers provably fast, condition-number-independent convergence for low-rank tensor estimation using t-SVD.
• A Novel Tensor Product-Based Neural Network for Solving Partial Differential Equations — TPNet solves PDEs and approximates functions via a tensor-product basis and direct least-squares, bypassing gradient training.
• Laplace Approximation for Bayesian Tensor Network Kernel Machines — LA-TNKM provides scalable, principled uncertainty estimates for tensor network kernel machines via Laplace approximation.
• Guided Trajectory Optimization with Sparse Scaling for Test-Time Diffusion — Reward-guided Trajectory Scaling (RTS) optimizes diffusion model noise paths via sparse, PCA-driven key-step selection and a coarse-to-fine search.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.