LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning

2026-06-11 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

LongSpike is a new Spiking Neural Network (SNN) framework designed to overcome the "memoryless" bottleneck of traditional SNNs in long-sequence tasks. Developed by Xinrui He, Qiyu Kang, Xuhao Li, and Zheng-Jun Zha, this model integrates fractional-order State-Space Modeling (f-SSM) from control theory into the spiking domain. Unlike dominant SNN architectures that rely on first-order Ordinary Differential Equations, LongSpike extends integer-order SSMs to the fractional-calculus regime, allowing for hierarchical integration of neuronal dynamics with long-memory kernels. To mitigate computational overhead and parallelization challenges typically associated with fractional operators, the framework utilizes a state-space formulation that supports efficient, parallel training. Empirical evaluations on benchmarks like Long Range Arena (LRA), WikiText-103, and Speech Commands demonstrate that LongSpike achieves superior accuracy compared to existing advanced SNNs, all while preserving sparse synaptic computation. The project's code is publicly available on GitHub.

Key takeaway

For AI Scientists and Machine Learning Engineers developing Spiking Neural Networks for long-sequence tasks, you should consider integrating fractional-order State-Space Modeling. This approach overcomes the "memoryless" limitations of first-order ODEs, allowing your models to capture complex, long-range dependencies more effectively. By utilizing a state-space formulation, you can achieve efficient, parallel training, leading to superior accuracy on benchmarks like Long Range Arena while maintaining sparse synaptic computation. Explore the LongSpike framework to enhance your SNNs' performance on challenging sequential data.

Key insights

LongSpike integrates fractional-order State-Space Modeling into SNNs to efficiently capture long-range dependencies, outperforming prior SNNs.

Principles

• First-order ODEs in SNNs create a "memoryless" bottleneck.
• Fractional-order dynamics enable hierarchical long-memory integration.
• State-space formulation can mitigate fractional operator overhead.

Method

LongSpike integrates fractional-order State-Space Modeling (f-SSM) into SNNs, extending integer-order SSMs to fractional calculus. It uses a state-space formulation for efficient, parallel training to handle computational overhead.

In practice

• Apply f-SSM to SNNs for long-sequence tasks.
• Use state-space formulation for parallel training of fractional operators.
• Evaluate SNNs on LRA, WikiText-103, and Speech Commands.

Topics

• Spiking Neural Networks
• Fractional-order Calculus
• State-Space Models
• Long Sequence Learning
• Long Range Arena
• Energy Efficiency

Code references

• xinruihe389-commits/LongSpike

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Stable Spike: Dual Consistency Optimization via Bitwise AND Operations for Spiking Neural Networks — Dual consistency optimization via "AND" operations and amplitude-aware noise improves SNN performance and generalization.
• Toward Compiler World Models: Learning Latent Dynamics for Efficient Tensor Program Search — Modeling tensor program optimization as latent dynamics improves efficiency and performance over static evaluation.
• Three factor delay learning rules for spiking neural networks — Online learning of synaptic and axonal delays significantly enhances SNN performance and efficiency for temporal tasks.
• Spike-Aware C++ INT8 Inference for Sparse Spiking Language Models on Commodity CPUs — Exploiting activation sparsity in spiking language models significantly boosts CPU inference throughput and reduces memory.
• Quantization of Spiking Neural Networks Beyond Accuracy — SNN quantization must preserve firing behavior, not just accuracy, for effective hardware deployment.
• Reformulating Neural Operators in $d+1$ Dimensions for Embedding Evolution — Neural operators redefined in d+1 dimensions using Schrödingerisation better capture system evolution and achieve superior performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.