Three factor delay learning rules for spiking neural networks

2026-02-13 · Source: cs.NE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, extended

Summary

A new method introduces three-factor learning rules for synaptic and axonal delays in Spiking Neural Networks (SNNs) and Spiking Recurrent Neural Networks (SRNNs), enabling online learning of these temporal parameters. This approach utilizes a smooth Gaussian surrogate for spike derivatives, exclusively for eligibility trace calculation, combined with a top-down error signal to update parameters. Experiments on the SHD speech recognition dataset demonstrate that incorporating delays improves accuracy by up to 20% over a weights-only baseline. When jointly learning weights and delays, the method achieves up to 14% higher accuracy for networks with similar parameter counts. Compared to existing offline backpropagation-based methods, this online learning approach achieves similar accuracy while reducing model size by 6.6x and inference latency by 67%, with only a 2.4% drop in classification accuracy. This benefits the design of power and area-constrained neuromorphic processors by facilitating on-device learning and reducing memory requirements.

Key takeaway

For AI Scientists and Research Scientists developing neuromorphic systems, integrating these three-factor delay learning rules offers a practical solution to enhance SNN performance on temporal tasks. You can achieve significant accuracy improvements (up to 20%) and efficiency gains (6.6x smaller models, 67% lower latency) compared to offline methods, especially in sparse or small network configurations. This enables on-device learning in memory-constrained environments without compromising accuracy.

Key insights

Online learning of synaptic and axonal delays significantly enhances SNN performance and efficiency for temporal tasks.

Principles

• Learnable delays improve temporal pattern recognition in SNNs.
• Three-factor learning rules enable online, real-time parameter updates.
• Sparsity amplifies the accuracy benefits of delay learning.

Method

The method uses three-factor learning rules with a smooth Gaussian surrogate for spike derivatives to enable online updates of synaptic and axonal delays in LIF-based SNNs and SRNNs, approximating BPTT performance.

In practice

• Implement online delay learning for resource-constrained neuromorphic systems.
• Consider joint weight and delay learning for improved accuracy-to-parameter ratio.
• Apply delay learning in sparse network configurations for enhanced benefits.

Topics

• Spiking Neural Networks
• Online Learning
• Three-Factor Learning Rules
• Synaptic and Axonal Delays
• Neuromorphic Computing

Best for: AI Scientist, Research Scientist, AI Researcher, Machine Learning Engineer, AI Hardware Engineer

Related on AIssential

• Stable Spike: Dual Consistency Optimization via Bitwise AND Operations for Spiking Neural Networks — Dual consistency optimization via "AND" operations and amplitude-aware noise improves SNN performance and generalization.
• Algorithm-hardware co-design of neuromorphic networks with dual memory pathways — A dual memory pathway architecture and co-designed hardware significantly enhance neuromorphic network efficiency and temporal processing.
• On the Evaluation of Spiking Neural Network Configurations for Network Intrusion Detection — Spike encoding scheme is more critical than neuron model for SNN-based intrusion detection performance.
• LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning — LongSpike integrates fractional-order State-Space Modeling into SNNs to efficiently capture long-range dependencies, outperforming prior SNNs.
• Neuromorphic Speech Enhancement with Dual-Branch Spiking Neural Networks — GSU-DBNet, a dual-branch SNN with gated spiking units, significantly improves neuromorphic speech enhancement performance and efficiency over ANNs.
• Quantization of Spiking Neural Networks Beyond Accuracy — SNN quantization must preserve firing behavior, not just accuracy, for effective hardware deployment.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.NE updates on arXiv.org.