Laplace Approximation for Bayesian Tensor Network Kernel Machines

2026-04-30 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

A novel Bayesian Tensor Network Kernel Machine (LA-TNKM) is introduced, leveraging a linearized Laplace approximation for Bayesian inference to provide principled uncertainty estimates in machine learning models. Traditional Gaussian Processes (GPs) offer robust uncertainty quantification but struggle with scalability beyond $10^5$–$10^6$ observations due to $\mathcal{O}(N^3)$ computational complexity. Tensor network kernel machines, while scalable, complicate standard probabilistic inference due to non-Gaussianity. LA-TNKM addresses this by approximating the posterior over model parameters with a Gaussian distribution, centered at a local maximum, where covariance captures local curvature. The method systematically evaluates various Hessian approximation techniques (Full, Generalized Gauss-Newton, Block-Diagonal, Diagonal, Last Core) and demonstrates competitive performance on UCI regression benchmarks, consistently matching or surpassing GPs and Bayesian Neural Networks (BNNs) in predictive accuracy and uncertainty calibration.

Key takeaway

For research scientists developing scalable probabilistic models, LA-TNKM offers a robust approach to uncertainty quantification in tensor network kernel machines. You should consider integrating LA-TNKM, particularly with the last-core or block-diagonal Hessian approximations, into your workflow to achieve competitive predictive performance and well-calibrated uncertainty estimates on medium- to large-scale datasets. This method provides a strong alternative to traditional GPs and BNNs, especially when dealing with high-dimensional data where scalability is a concern.

Key insights

LA-TNKM provides scalable, principled uncertainty estimates for tensor network kernel machines via Laplace approximation.

Principles

• Uncertainty estimation is critical for robust decision-making.
• Tensor networks enable scalable representation of high-dimensional data.
• Linearized Laplace approximation improves predictive performance.

Method

LA-TNKM uses a linearized Laplace approximation for Bayesian inference, approximating the posterior with a Gaussian and evaluating various Hessian approximations to balance memory, computation, and performance.

In practice

• Use Last or Block Hessian approximations for efficiency.
• Tune thresholding hyperparameter $\hat{t}$ via cross-validation.
• Linearized Laplace approximation consistently outperforms original.

Topics

• Laplace Approximation
• Tensor Network Kernel Machines
• Uncertainty Estimation
• Bayesian Inference
• Hessian Matrix Approximations

Code references

• AlbMLpy/laplace-tnkm

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Kernel-based guarantees for nonlinear parametric models in Bayesian optimization — A kernel-based framework provides theoretical guarantees for nonlinear parametric models in adaptive optimization.
• Low-Rank Tensor Completion Based on Fractional Regularization with Ky Fan p-k Norm — A novel nonconvex surrogate, TNPK, improves low-rank tensor completion by accurately approximating tubal rank with flexible parameters and proven convergence.
• Bayesian Poisson-Randomized Gamma Tensor Factorization with Application to International Trade Flows — A Bayesian Poisson-randomized Gamma tensor factorization models sparse, semi-continuous multiway data with excess zeros and heterogeneous dispersion.
• Uncertainty Estimation for Molecular Diffusion Models — A post-hoc method estimates per-sample uncertainty in molecular diffusion models using Laplace approximation to improve sample quality.
• Finite Neural Networks as Mixtures of Gaussian Processes: From Provable Error Bounds to Prior Selection — Finite neural networks can be approximated by Gaussian process mixtures with provable error bounds.
• The Value of Covariance Matching in Gaussian DDPMs and the Lanczos Sampler — Matching full posterior covariance in Gaussian DDPMs reduces path-KL error from Ω(1/T) to O(1/T^2), made practical by the Lanczos Gaussian sampler.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.