Reliability of Probabilistic Emulation of Physical Systems

2026-06-12 · Source: stat.ML updates on arXiv.org · Field: Science & Research — Mathematics & Computational Sciences, Engineering & Applied Sciences, Research Methodology & Innovation · Depth: Expert, extended

Summary

Researchers from The Alan Turing Institute systematically assessed the reliability of probabilistic emulation for physical systems, comparing generative models like diffusion or flow matching with ensembles of deterministic models trained using continuous ranked probability score (CRPS) loss. The study, using matched model sizes (around 80M parameters) and computational budgets (94 GPU-hours), evaluated both approaches across diverse 2D spatiotemporal systems including Advection-Diffusion, Gray-Scott, Conditioned Navier-Stokes, and Gross-Pitaevskii Equation. Findings indicate CRPS-trained ensembles generally provide more reliable uncertainty estimates, particularly in autoregressive rollouts, and offer significantly faster inference. While ambient space generative models can achieve comparable coverage, they incur much higher inference latency. The team released AutoCast and AutoSim to facilitate further research.

Key takeaway

If you are a Machine Learning Engineer developing probabilistic emulators for physical systems, prioritize CRPS-trained ensembles over latent space generative models. These ensembles consistently provide more reliable uncertainty estimates and significantly faster inference, crucial for real-world deployment and risk assessment. While ambient space generative models can match coverage, their high inference latency makes them less practical for high-dimensional problems. You should consider using the AutoCast framework to implement and benchmark these approaches effectively.

Key insights

CRPS-trained ensembles offer more reliable uncertainty quantification and faster inference than latent space generative models for physical system emulation.

Principles

• CRPS-trained ensembles yield more reliable UQ.
• Ambient space generative models improve coverage.
• Latent space size limits generative model performance.

Method

The study developed a framework to evaluate generative models and CRPS-trained ensembles on 2D spatiotemporal physical systems, assessing empirical coverage, accuracy, and computational efficiency under matched model size and budget.

In practice

• Use AutoCast for spatiotemporal forecasting.
• Generate datasets with AutoSim for prototyping.
• Employ Winkler score for CRPS checkpoint selection.

Topics

• Probabilistic Emulation
• Uncertainty Quantification
• CRPS-trained Ensembles
• Generative Models
• Spatiotemporal Forecasting
• AutoCast Framework

Code references

• alan-turing-institute/autosim
• alan-turing-institute/autocast
• probabilists/azula

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Assessing Emulator Design and Training for Modal Aerosol Microphysics Parameterizations in E3SMv2 — Emulator design choices, including scaling and network complexity, critically influence aerosol microphysics emulation accuracy in global atmospheric models.
• Machine learning for sustainable geoenergy — Machine learning can enhance sustainable geoenergy projects by addressing data scarcity and uncertainty through physics-informed methods.
• Phantom: Physics-Infused Video Generation via Joint Modeling of Visual and Latent Physical Dynamics — Integrating latent physical property inference into video generation improves physical plausibility and visual realism.
• Conditional Random Ordered Transport Spaces — Wasserstein distance alone cannot certify transformation admissibility; CROTS provides a framework for ordered distributional learning.
• Brief Announcement: Generative Markov Model for Distributed Computing Systems — A generative Markov model enables tractable simulation and optimization of complex distributed computing systems.
• From Possible to Probable AI Models — AI reliability hinges on distinguishing between what models *can* do and what they *consistently* do.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.