Doubly Outlier-Robust Online Infinite Hidden Markov Model

2026-02-06 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

This paper introduces the Batched Robust iHMM (BR-iHMM), a novel online infinite hidden Markov model designed to handle streaming data containing outliers and model misspecification. BR-iHMM achieves robustness by combining robust Generalized Bayes updates in the observation space with a batched inference mechanism that enforces robustness in the latent state space. This dual approach provides theoretical guarantees of bounded posterior influence, addressing the limitations of existing online iHMMs that are sensitive to outliers and prone to creating spurious regimes. The method balances adaptivity and robustness using two tunable parameters, and empirical evaluations across limit order book data, hourly electricity demand, and synthetic high-dimensional linear systems demonstrate that BR-iHMM reduces one-step-ahead forecasting error by up to 67% compared to competing online Bayesian methods, while also improving interpretability and computational efficiency.

Key takeaway

For Machine Learning Engineers developing online forecasting or regime detection systems in non-stationary environments, you should consider implementing BR-iHMM. Its dual robustness mechanism significantly improves predictive accuracy and state stability compared to traditional online iHMMs, especially when data contains outliers or experiences rapid regime changes. This approach can prevent the creation of spurious states and reduce forecasting errors by up to 67%, making your models more reliable and interpretable in production.

Key insights

Doubly robust online iHMMs combine observation-space and state-space robustness to mitigate outlier sensitivity and spurious regime creation.

Principles

• Robustness requires bounding influence in both observation and state spaces.
• Batching state inference prevents single outliers from dominating.
• Larger batch sizes increase robustness but may delay genuine regime detection.

Method

BR-iHMM uses Weighted Observation Likelihood Filter (WoLF) for observation-space robustness and a degenerate sticky HDP prior with batched state inference to enforce state-space robustness, implemented via particle learning.

In practice

• Apply BR-iHMM for forecasting in non-stationary, outlier-prone data streams.
• Optimize batch size (B) as a hyperparameter based on application needs.
• Consider BR-iHMM for high-dimensional exogenous input scenarios.

Topics

• Infinite Hidden Markov Models
• Outlier Robustness
• Regime Switching
• Generalized Bayesian Inference
• Batched Inference

Code references

• bayesian-optimization/BayesianOptimization

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Doubly Outlier-Robust Online Infinite Hidden Markov Model — BR-iHMM offers robust online learning for streaming data with outliers and model misspecification.
• VLBM: Variational Latent Basis Modeling for OOD Robust Multivariate Time Series Forecasting — VLBM improves multivariate time series forecasting by explicitly separating stable dynamics from OOD deviations using a latent basis model.
• OffQ: Taming Structured Outliers in LLM Quantization by Offsetting — OffQ effectively tames LLM activation outliers for W4A4KV4 quantization by converting them into absorbable offsets.
• Amortised and provably-robust simulation-based inference — NSM-Bayes offers provably robust and amortized simulation-based inference, overcoming outlier sensitivity in existing methods.
• Probabilistic data quality assessment for structural monitoring data via outlier-resistant conditional diffusion model — A conditional diffusion model improves SHM data quality assessment by robustly identifying and quantifying outliers.
• Distributionally Robust Set Representation Learning Under Inference-Time Element Corruption — SW-DRSO enhances set representation learning robustness by optimizing for worst-case inference-time corruption using a barycentric adversary.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.