Conditional neural control variates for variance reduction in Bayesian inverse problems

2026-02-26 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

Conditional Neural Control Variates (CNCV) is a modular method designed to reduce variance in Monte Carlo (MC) estimators for Bayesian inverse problems, particularly those constrained by partial differential equations (PDEs). It learns amortized control variates from joint model-data samples, generalizing across observations without retraining. To handle high-dimensional problems, CNCV employs an ensemble of hierarchical coupling layers, leveraging Stein's identity for tractable Jacobian trace computation. Training requires joint samples of unknown parameters and observed data, along with the posterior score function, which can be derived from physics-based likelihoods, neural operator surrogates, or learned generative models like conditional normalizing flows. The approach was validated on stylized inverse problems (Gaussian, Rosenbrock, nonlinear) and a PDE-constrained Darcy flow inverse problem, demonstrating substantial variance reduction. For instance, on a 100-dimensional Darcy flow problem, CNCV achieved a ~5.5x lower Mean Squared Error (MSE), effectively increasing sample size by 5.5 times.

Key takeaway

For research scientists working on Bayesian inverse problems with high computational costs due to Monte Carlo estimation, adopting Conditional Neural Control Variates (CNCV) can significantly reduce variance and improve estimation accuracy. You should consider integrating CNCV into your workflow, especially for PDE-constrained problems or when dealing with non-Gaussian posteriors, as it offers amortized variance reduction without requiring retraining for each new observation. This can lead to substantial computational savings and more reliable posterior expectation estimates.

Key insights

CNCV uses amortized neural control variates and Stein's identity to reduce Monte Carlo variance in Bayesian inverse problems.

Principles

• Amortized control variates generalize across observations.
• Stein's identity constructs zero-mean control variates.
• Ensemble averaging improves coverage and variance reduction.

Method

Train an ensemble of hierarchical coupling layers on joint samples and posterior scores. The architecture enables exact divergence computation for Stein's identity, minimizing the total variance of the controlled estimator.

In practice

• Integrates with simulation-based inference pipelines.
• Applicable to PDE-constrained problems with learned scores.
• Achieves ~5.5x effective sample increase for Darcy flow.

Topics

• Bayesian Inverse Problems
• Monte Carlo Variance Reduction
• Conditional Neural Control Variates
• Stein's Identity
• Hierarchical Coupling Layers

Code references

• luqigroup/groundwater
• luqigroup/grf
• luqigroup/cncv

Best for: Research Scientist, AI Researcher, AI Scientist, Machine Learning Engineer

Related on AIssential

• Variance Reduction for Non-Log-Concave Sampling with Applications to Inverse Problems — Unified analysis shows variance reduction improves sampling from non-log-concave distributions, enhancing convergence and sample quality.
• Neural Network Parameter-optimization of Gaussian Pre-marginalized Directed Acyclic Graphs — A new graphical structure and neural network duality enable robust parameter optimization for latent variable causal models.
• Variance Reduction for Expectations with Diffusion Teachers — CARV reduces Monte Carlo estimator variance in diffusion teacher pipelines via hierarchical sampling and amortization, boosting compute efficiency.
• Revisiting Neural Processes via Fourier Transform and Volterra Series — Combining Volterra series with frequency-domain convolutions yields scalable, interpretable translation-equivariant Neural Processes for irregular data.
• Amortised and provably-robust simulation-based inference — NSM-Bayes offers provably robust and amortized simulation-based inference, overcoming outlier sensitivity in existing methods.
• Tempered Sequential Monte Carlo for Trajectory and Policy Optimization with Differentiable Dynamics — TSMC optimizes trajectories and policies by framing controller design as inference, using tempered sampling and gradient-based rejuvenation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.