I built a Mamba1 variant I call SM1 with d_state=1 that runs on Blackwell in pure PyTorch [P]

2026-05-23 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, quick

Summary

A new Mamba1 variant, SM1 (Scalar Mamba1), has been developed, featuring a d_state=1 architecture that runs efficiently on Blackwell GPUs using pure PyTorch. SM1 replaces the complex selective scan with an exact closed-form solution implemented via two native PyTorch operations: torch.cumprod and torch.cumsum. This design significantly reduces memory footprint, eliminating the S dimension from Mamba1 scan intermediates, leading to 16x less scan memory compared to a d_state=16 Mamba1. For a 130M parameter model, SM1's inference state is only 56 KB (14,080 floats), offering O(1) memory per token without a KV cache. The developer is currently training a 130M parameter model on 163K MIDI files (2.5B tokens), which fits comfortably within half of a 16 GB RTX 5060 Ti card. The approach suggests that a scalar d_state is sufficient when structural information is already encoded in tokens.

Key takeaway

For Machine Learning Engineers optimizing model deployment on consumer-grade GPUs, SM1 presents a compelling alternative to standard Mamba architectures. If you are struggling with memory constraints on cards like the RTX 5060 Ti, consider implementing d_state=1 Mamba variants. This approach significantly reduces inference state memory to O(1) per token, potentially enabling larger models to fit within your existing hardware without a KV cache. Explore the provided GitHub repository for a practical implementation.

Key insights

SM1 offers a memory-efficient Mamba1 variant with d_state=1 using native PyTorch for O(1) inference.

Principles

• d_state=1 enables exact closed-form recurrence solutions.
• Token-encoded structure can reduce d_state requirements.
• Native PyTorch ops can replace complex custom kernels.

Method

SM1 implements the d_state=1 selective scan using torch.cumprod for L and torch.cumsum for h, followed by y = h * C, providing an exact closed-form solution.

In practice

• Deploy Mamba1 variants on resource-constrained GPUs.
• Explore d_state=1 for structured token data.
• Implement custom scans with standard PyTorch functions.

Topics

• Mamba Architecture
• State Space Models
• PyTorch Implementation
• GPU Memory Optimization
• O(1) Inference
• MIDI Processing

Code references

• CopilotCoding/MidiMamba

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• Meet Mamba-3: A New State Space Model Frontier with 2x Smaller States and Enhanced MIMO Decoding Hardware Efficiency — Mamba-3 advances State Space Models through novel discretization, complex-valued updates, and MIMO formulation for efficiency and accuracy.
• A Practical Guide to PyTorch: The Deep Learning Framework Built for People Who Think in Code — PyTorch's dynamic computation graphs and Python integration empower flexible deep learning research and deployment.
• The Past Is Prologue: A Plug-in Controller for Selective Updates in Sequentially Evolving LLM Memory — Janus selectively validates LLM memory updates to prevent degradation, improving accuracy by +2.7 to +4.6 points.
• On Sequence-to-Sequence Models for Automated Log Parsing — Mamba and Transformer models offer superior log parsing accuracy and sample efficiency, with Mamba providing cost advantages.
• MambaADv2: Evolving Duality-enhanced State Space Model for Unsupervised Anomaly Detection — MambaADv2 employs duality-enhanced state space models for efficient, multi-class unsupervised anomaly detection, addressing CNN and Transformer limitations.
• OPRD: On-Policy Representation Distillation — OPRD improves on-policy distillation by aligning hidden states, eliminating sampling variance and providing richer structural information.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.