Equilibrium Propagation and Hamiltonian Inference in the Diffusive Fitzhugh-Nagumo Model

2026-05-23 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences, Life Sciences & Biology · Depth: Expert, extended

Summary

This work extends the Equilibrium Propagation (EqProp) framework to skew-gradient systems, demonstrating an equivalence between deep Energy-Based Models (EBMs) and Hamiltonian neural networks. Focusing on diffusively coupled Fitzhugh-Nagumo (FHN) neurons as a prototype, the research shows that EqProp can be applied to FHN networks because their stationary solutions are described by self-adjoint operators. For FHN networks structured as deep residual networks, steady-state solutions admit a spatial Hamiltonian, allowing for the application of Hamiltonian Echo Backpropagation (HEB). The authors derive explicit layer-wise Hamiltonian recurrence relations for inference in both deep FHN networks and deep EBMs, enabling single-pass inference given initial momentum, contrasting with the iterative convergence typically required by EBMs. Experimental training of a 5-layer FHN network on MNIST using EqProp, with a beta_nudge of 0.9 and weight initialization scale of 0.014, validates the theoretical analysis. Simulations on a 30-layer, 64-neuron FHN network confirm the equivalence of time-based and Hamiltonian spatial integration.

Key takeaway

For AI Scientists and Research Scientists developing biologically plausible learning algorithms or seeking more efficient inference in energy-based models, this work provides a crucial theoretical and practical bridge. You should investigate applying Equilibrium Propagation to skew-gradient systems like Fitzhugh-Nagumo networks for local credit assignment. Furthermore, consider leveraging the derived Hamiltonian recurrence relations to achieve single-pass, feedforward inference in deep Energy-Based Models, potentially overcoming the computational expense of iterative convergence.

Key insights

Equilibrium Propagation and Hamiltonian Echo Backpropagation can be extended to skew-gradient systems and deep Energy-Based Models for efficient credit assignment and inference.

Principles

• Self-adjointness enables local credit assignment in gradient systems.
• Skew-gradient systems, at steady state, can exhibit self-adjoint properties.
• Deep Energy-Based Models have a feedforward Hamiltonian formulation.

Method

The method involves extending Equilibrium Propagation to stationary solutions of skew-gradient systems and deriving layer-wise Hamiltonian recurrence relations for single-pass inference in deep Fitzhugh-Nagumo networks and deep Energy-Based Models, converting a two-point boundary problem into an initial-value problem.

In practice

• Train deep Fitzhugh-Nagumo networks on MNIST using Equilibrium Propagation.
• Utilize Hamiltonian recurrence for single-pass inference in deep EBMs.
• Initialize FHN parameters in the Turing pattern forming regime.

Topics

• Equilibrium Propagation
• Hamiltonian Neural Networks
• Fitzhugh-Nagumo Model
• Energy-Based Models
• Skew-Gradient Systems
• Biologically Plausible Learning
• Inference Optimization

Code references

• jackdkendall/Self-Adjoint-Learning

Best for: AI Scientist, Research Scientist

Related on AIssential

• Hybridizing Equilibrium Propagation with Ising Machines for Efficient Energy-Based Learning — Hybridizing EP with Ising dynamics improves energy-based learning by overcoming local minima and accelerating convergence.
• Thermodynamic Diffusion Inference with Minimal Digital Conditioning — Thermodynamic diffusion inference can achieve massive energy savings by encoding U-Net skip connections and overcoming input conditioning.
• Uncovering Symmetry Transfer in Large Language Models via Layer-Peeled Optimization — LLM training implicitly transfers data symmetries into structured geometric configurations within model parameters.
• Hebbian-Oscillatory Co-Learning — HOC-L unifies structural plasticity and phase synchronization in sparse neural networks via synchronization-gated Hebbian learning.
• Attention is Just Another Name for Coupling?: A Fast-Slow ODE Perspective on Hierarchical Pretraining — A hierarchical fast-slow ODE architecture for pretraining combines token-rate attention with a slower, pooled-token path, mapping equilibrium to stationary distributions.
• Gradient descent at the Edge of Stability: free energy model and kinetic description of the two-layer network — A new model tracks average trajectory and oscillation covariance, revealing an effective free energy for Edge of Stability dynamics.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.