Regularity and Stability Properties of Selective SSMs with Discontinuous Gating

2026-07-08 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

Nikola Zubić, Davide Scaramuzza, and their team investigate the stability and regularity of continuous-time deep selective State-Space Models (SSMs), such as Mamba, HGRN, and GLA, which feature input-dependent, time-varying parameters and potentially discontinuous gating signals. Leveraging passivity and Input-to-State Stability (ISS) theories, their work establishes that intrinsic energy dissipation ensures exponential forgetting of past states (Theorem 3.1). They prove that unforced system dynamics possess a minimal quadratic energy function whose defining matrix exhibits robust AUC_loc regularity, accommodating discontinuous gating (Theorem 3.3). Furthermore, assuming universal quadratic passivity leads to parametric LMI conditions and a crucial kernel constraint: "energy-less" state directions must be output-unobservable under any gating (Theorem 4.2), formalizing "irreversible forgetting" (Theorem 4.3). Finally, the research provides sufficient conditions for global ISS, linking uniform local dissipativity to overall system robustness (Theorem 5.1).

Key takeaway

For AI Architects designing or evaluating selective SSMs like Mamba, understanding these stability guarantees is crucial. You should prioritize initialization strategies that establish a favorable baseline energy landscape, promoting stable learning and controlled memory decay. Ensure your gating mechanisms adhere to derived LMI and kernel constraints to maintain robust passivity and prevent "un-forgetting" of energy-less states, thereby enhancing model reliability and predictability.

Key insights

Selective SSMs' stability and memory properties are rigorously characterized by passivity and ISS, even with discontinuous gating.

Principles

• Intrinsic energy dissipation guarantees exponential state forgetting.
• Universal quadratic passivity constrains gating mechanisms.
• "Energy-less" state directions become unobservable.

Method

The paper applies passivity theory and Input-to-State Stability (ISS) to continuous-time selective SSMs, deriving LMI conditions and kernel constraints for stability analysis.

In practice

• Design stable SSMs by ensuring LMI conditions.
• Initialize SSMs to establish favorable energy landscapes.
• Understand memory decay in SSMs via exponential forgetting.

Topics

• Selective SSMs
• Passivity Theory
• Input-to-State Stability
• Lyapunov Stability
• Discontinuous Gating
• Model Stability

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Architect

Related on AIssential

• PIMSM: Physics-Informed Multi-Scale Mamba for Stable Neural Representations under Distribution Shift — Anchoring model memory to physical timescales via spectrum-derived parameters enhances robustness to distribution shifts.
• On the Stability of Nonlinear Dynamics in GD and SGD: Beyond Quadratic Potentials — Nonlinear dynamics significantly impact GD/SGD stability, often contradicting linear approximations, especially with high-order derivatives and single batch instability.
• Coercivity and Local Convergence of Physical Learning in Linear Circuits — This work provides the first local convergence analysis for physical learning methods in linear circuits under a coercivity condition.
• Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies — Forgetting in LLM fine-tuning with Evolution Strategies is performance drift, not irreversible, and is largely avoidable using Anchored Weight Decay.
• Beyond Expected Information Gain: Stable Bayesian Optimal Experimental Design with Integral Probability Metrics and Plug-and-Play Extensions — IPM-based BOED offers superior stability and computational efficiency over traditional KL-divergence methods, especially in high-dimensional settings.
• Stable Deep Reinforcement Learning via Isotropic Gaussian Representations — Isotropic Gaussian representations fundamentally stabilize deep RL by mitigating non-stationarity, preventing collapse, and reducing neuron dormancy.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.