PIMSM: Physics-Informed Multi-Scale Mamba for Stable Neural Representations under Distribution Shift

2026-05-19 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Life Sciences & Biology · Depth: Expert, extended

Summary

Physics-Informed Multi-Scale Mamba (PIMSM) is a novel state-space architecture designed to improve the robustness and representation stability of scientific foundation models under distribution shifts. Traditional sequence backbones often fail to preserve the multiscale temporal structure inherent in natural dynamical systems, leading to "temporal kernel mismatch" and degraded transfer performance. PIMSM addresses this by mapping spectrum-estimated transition points (knee frequencies) between frequency regimes to scale-specific discretization parameters, anchoring them to acquisition time units. Evaluated on Human Connectome Project (HCP) fMRI data, PIMSM demonstrated improved robustness and stability under severe temporal-context truncation, extreme low-resource transfer, and resting-state to task-state generalization. The architecture also achieved the lowest variable-wise Mean Absolute Error (MAE) across all reported horizons and variables on Weather-5K held-out-station spatial out-of-distribution forecasting, without modality-specific adaptation. These results highlight temporal-scale alignment as a practical inductive bias for scientific foundation models.

Key takeaway

For AI Scientists and Machine Learning Engineers developing foundation models for scientific time series, incorporating physics-informed multiscale temporal parameterization is crucial. Your models will exhibit greater robustness and representation stability under deployment shifts, such as truncated data or domain changes, by explicitly aligning temporal kernels with the signal's physical organization. Consider integrating spectral analysis to guide architectural choices, ensuring your models preserve structure rather than merely fitting correlations.

Key insights

Anchoring model memory to physical timescales via spectrum-derived parameters enhances robustness to distribution shifts.

Principles

• Multiscale temporal structure is critical for scientific time series.
• Temporal kernel mismatch causes representation drift under shift.
• Physics-informed inductive biases improve model transferability.

Method

PIMSM uses SpectralHyperNet to estimate knee frequencies from input spectra, mapping these to scale-specific Mamba discretization parameters (Δk) anchored to acquisition units, and regularizing the A-scale to prevent compensation.

In practice

• Use spectrum analysis to identify knee frequencies.
• Anchor model discretization steps to physical acquisition units.
• Apply cross-scale attention for multi-scale state interaction.

Topics

• Physics-Informed Mamba
• Multi-Scale State-Space Models
• Distribution Shift Robustness
• Temporal Kernel Mismatch
• Spectral Knee Frequencies

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Physics-conforming Latent Twins — Physics-conforming Latent Twins enforce physical principles in latent space, improving surrogate model fidelity and long-term behavior for time-dependent systems.
• SSMamba: A Self-Supervised Hybrid State Space Model for Pathological Image Classification — Task-specific architectural designs and in-domain SSL significantly enhance pathological image analysis performance over generic FMs.
• Spectra: Rethinking Optimizers for LLMs Under Spectral Anisotropy — LLM gradients exhibit a "spike-tail" anisotropy where dominant linguistic structures suppress long-tail semantic learning.
• Identifiability Without Gaussianity: Symbolic World Models and Near-Infinite Temporal Consistency — Symbolic grounding in causal dynamics enables near-infinite temporal consistency in World Models, surpassing statistical models' Gaussian limitations.
• Robust Parameter and State Estimation in Multiscale Neuronal Systems Using Physics-Informed Neural Networks — PINNs offer robust parameter and state estimation in multiscale neuronal systems, overcoming limitations of traditional methods.
• MAMVI: 3D Test-Time Adaptation via Masked Multi-View Point Clouds — MAMVI unifies multi-view 3D test-time adaptation into a single, faster step using hybrid masking and multi-view consensus.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.