Amortized Filtering and Smoothing with Conditional Normalizing Flows

2026-04-09 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

This paper introduces AFSF, a unified amortized framework for Bayesian filtering and smoothing in high-dimensional nonlinear dynamical systems, leveraging conditional normalizing flows and recurrent neural networks. AFSF encodes observation history into a fixed-dimensional summary statistic using a recurrent encoder. This shared representation then conditions both a forward flow, approximating the filtering distribution $p(u_{t}\mid y_{1:t})$, and a backward flow, approximating the backward transition kernel $p(u_{t-1}\mid u_{t},y_{1:t-1})$. The smoothing distribution is recovered by combining the terminal filtering distribution with the learned backward flow via standard backward recursion. The framework supports extrapolation beyond training horizons and improves trajectory-level smoothing through implicit regularization. A flow-based particle filtering variant is also developed for alternative online inference and ESS-based diagnostics. Numerical experiments on an advection-diffusion system, a stochastic volatility model, a PDE system, and Lorenz systems demonstrate AFSF's accuracy and robustness.

Key takeaway

For Machine Learning Engineers developing real-time state estimation systems, AFSF offers a robust approach to handling high-dimensional, nonlinear dynamics. Your teams should consider integrating this amortized framework to achieve accurate filtering and smoothing, especially when dealing with complex systems where traditional Gaussian or Monte Carlo methods struggle. The framework's ability to extrapolate beyond training data and its implicit regularization for trajectory-level smoothing can significantly enhance model performance and reliability in production environments.

Key insights

AFSF unifies filtering and smoothing for high-dimensional nonlinear systems via amortized conditional normalizing flows and shared summary statistics.

Principles

• Shared summary statistics regularize latent state trajectories.
• Amortized inference enables fast posterior sampling at test time.
• Flow-based particle filters can reduce weight degeneracy.

Method

AFSF uses a recurrent encoder for observation history summaries, then trains a forward flow for filtering and a backward flow for the backward transition kernel, both conditioned on these summaries. Smoothing is achieved via backward recursion.

In practice

• Apply AFSF for high-dimensional state estimation in nonlinear systems.
• Use flow-based particle filtering for online inference with diagnostics.
• Consider shared summary networks for improved smoothing consistency.

Topics

• Amortized Filtering and Smoothing
• Conditional Normalizing Flows
• Recurrent Neural Networks
• Bayesian Inference
• High-Dimensional Dynamical Systems

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• Smoothness-Based Derandomization of PAC-Bayes Bounds — PAC-Bayes derandomization for smooth loss functions quantifies the cost of deterministic predictors using flatness metrics.
• Flow Matching for Efficient and Scalable Data Assimilation — EnFF uses flow matching to create a training-free, scalable data assimilation framework with improved cost-accuracy.
• Two-Layer Linear Auto-Regressive Models Estimate Latent States — Two-layer linear auto-regressive models approximate Kalman filtering for latent state estimation in partially observed linear dynamical systems.
• Efficient Online Prediction for High-Dimensional Time Series via Joint Tensor Tucker Decomposition — TOPA-AAW offers efficient, accurate online prediction for high-dimensional streaming tensor time series via adaptive tensor factorization.
• Amortized mean-shift interacting particles — Amortized mean-shift interacting particles efficiently integrate posteriors from samples, avoiding costly density or score evaluations.
• Neural Galerkin Normalizing Flows for Bayesian Inference of Diffusions with Inaccessible Boundaries — Neural Galerkin Normalizing Flows efficiently learn diffusion transition densities for Bayesian inference, bypassing complex analytical challenges.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.