Pulse-Driven Neural Architecture: Learnable Oscillatory Dynamics for Robust Continuous-Time Sequence Processing

2026-03-03 · Source: cs.NE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Data Science & Analytics · Depth: Expert, extended

Summary

PDNA (Pulse-Driven Neural Architecture) augments continuous-time recurrent networks, specifically Closed-form Continuous-time (CfC) networks, with learnable oscillatory dynamics to enhance robustness against input interruptions. The architecture introduces a "pulse module" generating structured oscillations $A\cdot\sin(\omega t+\varphi(h))$ with learnable frequencies and state-dependent phase, and a "self-attend module" applying recurrent self-attention to the hidden state. An ablation study on sequential MNIST (sMNIST) with five random seeds demonstrated that these structured oscillatory dynamics significantly improve robustness to input gaps. The pulse variant achieved a 4.62 percentage point (pp) multi-gap accuracy advantage over the baseline (92.86% vs. 88.24%), with a large effect size (Cohen's $d=0.87$). The self-attend variant showed a 2.78 pp multi-gap advantage ($p=0.041$). A noise control, adding random perturbations of equal magnitude, provided no benefit, confirming the structural nature of PDNA's advantage. The learned pulse strength parameter \alpha increased approximately 66 times from its initial value, and the effective pulse magnitude \alpha\cdot\|A\|_2 increased about 420 times, indicating active utilization of the oscillatory mechanism. PDNA adds 38% more parameters but only 5% wall-time overhead.

Key takeaway

For research scientists developing robust sequence models, you should consider integrating biologically-inspired oscillatory dynamics into continuous-time architectures. PDNA's approach, which significantly improves performance under multi-gap conditions with minimal computational overhead, suggests that internal, learnable temporal mechanisms are critical for maintaining state and information coherence when external inputs are intermittent. Explore how state-dependent phase and diverse learned frequencies can enhance your model's resilience to real-world data imperfections.

Key insights

Biologically-inspired oscillatory dynamics significantly improve neural network robustness to temporal input gaps.

Principles

• Structured oscillations provide temporal encoding.
• State-dependent phase maintains oscillation coherence.
• Learnable dynamics are crucial for effective augmentation.

Method

PDNA augments CfC networks with a pulse module for learnable oscillations and a self-attend module for recurrent self-attention, trained end-to-end to maintain state during input gaps.

In practice

• Implement learnable oscillatory dynamics for gap robustness.
• Use state-dependent phase for context-sensitive oscillation.
• Evaluate models with gapped test data, not just standard metrics.

Topics

• Pulse-Driven Neural Architecture
• Learnable Oscillatory Dynamics
• Continuous-Time Networks
• Temporal Robustness
• Recurrent Self-Attention

Code references

• Parassharmaa/pdna

Best for: Research Scientist, AI Researcher, AI Scientist, Machine Learning Engineer

Related on AIssential

• Hebbian-Oscillatory Co-Learning — HOC-L unifies structural plasticity and phase synchronization in sparse neural networks via synchronization-gated Hebbian learning.
• Separable neural architectures as a primitive for unified predictive and generative intelligence — Separable Neural Architectures unify diverse models by exploiting factorisable structures for predictive and generative intelligence.
• Self-orthogonalizing attractor neural networks emerging from the free energy principle — Attractor networks emerge from the Free Energy Principle, yielding self-organizing, orthogonal representations and robust learning.
• Attention as Frustrated Synchronization — Frustrated synchronization enables attention to predict context continuation, not just retrieve consensus.
• Lifecycle-Aware Federated Continual Learning in Mobile Autonomous Systems — A dual-timescale FCL framework improves autonomous system adaptation by preventing immediate forgetting and recovering from long-term drift.
• Structure Over Nonlinearity: Explicit Interaction Architectures for Dynamical Learning — Dynamical system learning benefits from explicit interaction structures more than complex nonlinearities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.NE updates on arXiv.org.