Amortised and provably-robust simulation-based inference

2026-02-13 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

Ayush Bharti, Charita Dellaporta, Yuga Hikida, and François-Xavier Briol introduce a novel simulation-based inference (SBI) approach, Neural Score-Matching Bayes (NSM-Bayes), designed to be amortized and provably robust to outliers in data. Existing SBI methods, while widely used in fields like particle physics and epidemiology, often fail to account for extreme values caused by measurement errors or human mistakes, and typically lack formal robustness guarantees or amortized inference capabilities. NSM-Bayes addresses these limitations by leveraging generalized Bayesian inference and a neural approximation of a weighted score-matching loss. The method offers two versions: a general NSM-Bayes and a conjugate special case, NSM-Bayes-conj, which significantly reduces computational costs by eliminating the need for Markov chain Monte Carlo (MCMC) sampling. The authors provide theoretical proofs demonstrating the lack of robustness in Neural Likelihood Estimation (NLE) and the global-bias robustness of both NSM-Bayes and NSM-Bayes-conj, particularly when using inverse multi-quadratic (IMQ) weight functions.

Key takeaway

Research Scientists working with complex simulator-based models and real-world data should consider adopting NSM-Bayes or NSM-Bayes-conj to mitigate the impact of outliers. Your current Neural Likelihood Estimation (NLE) methods are likely susceptible to data contamination, leading to biased posterior approximations. Implementing NSM-Bayes provides formal robustness guarantees and, with the conjugate version, offers significant computational advantages by avoiding MCMC sampling, ensuring more reliable and efficient inference in the presence of noisy data.

Key insights

NSM-Bayes offers provably robust and amortized simulation-based inference, overcoming outlier sensitivity in existing methods.

Principles

• Outliers significantly bias standard neural likelihood estimation.
• Weighted score-matching loss enhances robustness.
• Conjugate models can eliminate MCMC for efficiency.

Method

NSM-Bayes trains a neural conditional density model on simulated data, then updates prior beliefs using a weighted score-matching loss, enabling amortized and robust inference without MCMC in its conjugate form.

In practice

• Use IMQ functions for robust data weighting.
• Calibrate learning rates for reliable posterior coverage.
• Consider NSM-Bayes-conj for computational efficiency.

Topics

• Simulation-Based Inference
• Robust Bayesian Inference
• Score Matching
• Amortized Inference
• Neural Likelihood Estimation

Best for: Research Scientist, AI Researcher, AI Scientist, Machine Learning Engineer

Related on AIssential

• Asymptotically Log-Optimal Bayes-Assisted Confidence Sequences for Bounded Means — Bayes-assisted confidence sequences leverage prior information for efficiency without sacrificing anytime-valid coverage.
• Robust Simulation-Based Inference Under Missing Data via Neural Processes — RISE jointly learns data imputation and parameter inference in SBI to robustly handle missing data and prevent biased posteriors.
• Neural Bayesian Anomaly Mitigation: A Robust Loss that Doubles as an Unsupervised Contamination Classifier — NBAM is a robust loss that simultaneously classifies contamination using a Bayesian latent-switch mixture model.
• XMSE-Aware Adaptive Empirical Bayes Estimation — An XMSE-aware mixed estimator adaptively combines ML and Empirical Bayes to mitigate kernel misspecification risks.
• Realistic noise synthesis reduces bias and improves tissue microstructure estimation with supervised machine learning — Realistic noise synthesis in simulated data is crucial for unbiased dMRI microstructure estimation, especially at low SNR.
• On the Robustness of Kernel Goodness-of-Fit Tests — Existing kernel goodness-of-fit tests lack robustness, necessitating a new framework for practical model validation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.