Simulation-Based Empirical Bayes

2026-07-27 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

Simulation-based empirical Bayes (SBEB) is a novel method for simultaneous inference across related latent variables when likelihoods are only available through a simulator. It extends classical empirical Bayes (EB) and simulation-based inference (SBI) by iteratively refining a nonparametric EB prior within an SBI loop. SBEB computes EB estimates without an explicit density, utilizing observed data, simulator samples, and an amortized inference network. The method demonstrates improved accuracy over fixed-prior SBI in various scientific simulators, including linear Gaussian, nonlinear static, oscillator, predator-prey, and evolutionary models. It also shows benefits in real-world applications like heterogeneous product choice and measles outbreak dynamics across 40 UK cities from 1944 to 1965.

Key takeaway

For research scientists developing models with intractable likelihoods and multiple related units, SBEB offers a robust approach to improve inference accuracy. You should consider implementing SBEB when a common population prior exists, as it learns this prior from data, outperforming fixed-prior SBI, especially with sufficient unit-level signal. Be prepared for increased computational cost and the need for an expressive amortized inference family.

Key insights

SBEB combines empirical Bayes and simulation-based inference to learn priors for intractable likelihoods.

Principles

• The SBEB update improves the fit of the induced marginal distribution.
• Prior KL divergence decreases, implying marginal convergence to the population.
• Prior identifiability ensures marginal convergence leads to prior convergence.

Method

SBEB iteratively approximates posteriors, averages them to refine an empirical Bayes prior, samples from this prior, simulates new data, and refits an amortized inference network.

In practice

• Infer city-specific epidemic parameters from measles time series.
• Estimate person-specific preference vectors in discrete choice models.
• Recover latent variables in complex scientific simulators.

Topics

• Empirical Bayes
• Simulation-Based Inference
• Intractable Likelihoods
• Amortized Inference
• Population Prior
• Epidemic Modeling
• Discrete Choice Models

Best for: AI Scientist, Research Scientist

Related on AIssential

• Simulation-Based Empirical Bayes — SBEB extends Empirical Bayes to implicit likelihoods via simulation-based inference, improving accuracy.
• An Introduction to Bayesian and Frequentist Simulation-Based Inference with Machine Learning — Machine learning-based Simulation-based inference (SBI) offers a versatile approach to inverse problems within both Bayesian and frequentist frameworks.
• Amortised and provably-robust simulation-based inference — NSM-Bayes offers provably robust and amortized simulation-based inference, overcoming outlier sensitivity in existing methods.
• XMSE-Aware Adaptive Empirical Bayes Estimation — An XMSE-aware mixed estimator adaptively combines ML and Empirical Bayes to mitigate kernel misspecification risks.
• Robust Sequential Experimental Design for A/B Testing — Robust sequential experimental design improves A/B testing efficiency by mitigating model misspecification and finite-sample imbalances.
• Amortized Inference for Correlated Discrete Choice Models via Equivariant Neural Networks — An equivariant neural network emulator enables rapid, accurate estimation of discrete choice probabilities for models with complex correlated errors.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.