HAMSA: Scanning-Free Vision State Space Models via SpectralPulseNet

2026-04-16 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

HAMSA is a novel Vision State Space Model (SSM) that operates directly in the spectral domain, eliminating the complex scanning strategies used by existing SSMs like Vim, VMamba, and SiMBA. It achieves this through three innovations: a simplified kernel parameterization using a single Gaussian-initialized complex kernel, SpectralPulseNet (SPN) for input-dependent frequency gating, and a Spectral Adaptive Gating Unit (SAGU) for stable gradient flow. By employing FFT-based convolution, HAMSA maintains O(L log L) complexity. On ImageNet-1K, HAMSA achieves 85.7% top-1 accuracy, outperforming other SSMs, and demonstrates 2.2X faster inference than transformers (4.2ms vs 9.2ms for DeiT-S). It also offers 1.4-1.9X speedup over scanning-based SSMs, uses less memory (2.1GB vs 3.2-4.5GB), and consumes less energy (12.5J vs 18-25J), while showing strong generalization across various tasks.

Key takeaway

For AI engineers and research scientists developing vision models, HAMSA presents a compelling alternative to traditional scanning-based SSMs and transformers. Its spectral domain processing and simplified architecture offer significant improvements in inference speed, memory footprint, and energy efficiency. You should consider evaluating HAMSA for computer vision tasks requiring high performance and resource optimization, especially for deployment on constrained hardware.

Key insights

HAMSA introduces a scanning-free Vision SSM operating in the spectral domain for improved efficiency and performance.

Principles

• Spectral domain processing can simplify SSMs.
• Input-dependent frequency gating enhances adaptability.

Method

HAMSA uses a single Gaussian-initialized complex kernel, SpectralPulseNet for frequency gating, and Spectral Adaptive Gating Unit for stable gradients, all leveraging FFT-based convolution.

In practice

• Achieves 85.7% top-1 accuracy on ImageNet-1K.
• 2.2X faster inference than DeiT-S transformers.
• Reduces memory to 2.1GB and energy to 12.5J.

Topics

• Vision State Space Models
• SpectralPulseNet
• Spectral Adaptive Gating Unit
• FFT-based Convolution
• ImageNet-1K

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Linear Recurrent Unit with Semantic Modulation for Image Super-Resolution — A Linear Recurrent Unit (LRU) combined with a Semantic Modulating Unit (SMU) significantly enhances single-image super-resolution performance and efficiency.
• FrequencyFormer: A Co-Designed Sensor-to-Processor Pipeline for Frequency-Domain Vision Transformer Inference — Frequency-domain tokenization at the sensor-edge dramatically reduces data movement and energy for Vision Transformer inference, enabling efficient deployment.
• Sessa: Selective State Space Attention — Sessa integrates attention into a recurrent feedback path, achieving superior long-range memory compared to Transformers and Mamba.
• RPCASSM: Robust PCA State Space Model For Infrared Small Target Detection — RPCASSM leverages Robust PCA to design specialized state space modules for background and target, accurately detecting infrared small targets by their spatial properties.
• Spatial-Aware Reduction Framework: Towards Efficient and Faithful Visual State Space Models — Token reduction in visual State Space Models requires spatial awareness to prevent performance collapse.
• MOSA: Motion-Guided Semantic Alignment for Dynamic Scene Graph Generation — MoSA improves dynamic scene graph generation by integrating motion features and semantic alignment.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.