CLT-Optimal Parameter Error Bounds for Linear System Identification

2026-04-24 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

This paper introduces sharpened parameter error bounds for identifying discrete-time linear dynamical systems (LDS) using ordinary least-squares (OLS) regression. The authors demonstrate that existing state-of-the-art bounds often overstate the squared parameter error, in both spectral and Frobenius norms, by a factor of the system's state-dimension. To address this, they propose a novel second-order decomposition of the parameter error, where a matrix-valued martingale term accurately captures the Central Limit Theorem (CLT) scaling. This new analysis yields finite-sample bounds for both stable systems and the many-trajectories setting. These bounds match instance-specific optimal rates up to constant factors in Frobenius norm and polylogarithmic state-dimension factors in spectral norm, significantly improving upon previous results that exhibit gaps related to noise covariance and non-uniform state covariance matrices.

Key takeaway

For AI Scientists and Research Scientists working on system identification, these refined OLS parameter error bounds offer a more accurate assessment of model performance. You should re-evaluate your expected error margins, especially in scenarios with non-isotropic noise or non-uniform state covariance matrices, as previous bounds likely overestimated the true error. Incorporating these sharper bounds can lead to more precise data requirements for learning accurate models and better resource allocation for system identification tasks.

Key insights

Existing OLS parameter error bounds for LDS identification overstate error by a state-dimension factor, necessitating sharper analysis.

Principles

• Minimax-optimal rates are not universally sharp across all problem instances.
• Asymptotic normality can reveal instance-specific optimal scaling.
• Second-order error decomposition improves bound accuracy for matrix-valued martingales.

Method

A novel second-order decomposition of OLS parameter error is used, where the lower-order term is a matrix-valued martingale capturing CLT scaling, analyzed via noncommutative Burkholder inequalities.

In practice

• Use new bounds to more accurately predict OLS error in LDS.
• Apply to stable systems and multi-trajectory scenarios.
• Consider noise and state covariance non-uniformity in error estimation.

Topics

• Linear System Identification
• Ordinary Least-Squares Regression
• Parameter Error Bounds
• Asymptotic Normality
• Frobenius and Operator Norms

Best for: AI Scientist, Research Scientist

Related on AIssential

• Upper Bounds on the Generalization Error of Deep Learning Models via Local Robustness and Stability — A new generalization bound improves accuracy by locally scaling robustness based on stable and unstable samples.
• Conformal Candidate Certification for Offline Model-Based Optimization — Conformal Candidate Certification (CCC) provides per-candidate lower bounds for offline MBO designs, addressing surrogate overestimation in OOD regions.
• Wasserstein bounds for denoising diffusion probabilistic models via the F\"ollmer process — DDPM sampling errors in 2-Wasserstein distance are bounded by reinterpreting the process as a Föllmer discretization.
• Error Analysis for Deep ReLU Feedforward Density-Ratio Estimation with Bregman Divergence — BDD density-ratio estimators achieve minimax optimal error bounds and mitigate the curse of dimensionality.
• Limits of spectral learning under noise — Spectral learning exhibits a fundamental noise threshold that limits functional structure recovery from noisy data.
• False Sense of Safety in Selective Signal Classification: Auditing Bound Tightness and Exchangeability for Risk Control — Selective prediction's safety promise is often undermined by uncertified methods or broken exchangeability in grouped deployments.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.