Closed-form predictive coding via hierarchical Gaussian filters

2026-05-22 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new approach to predictive coding (PC) networks, submitted on May 19, 2026, addresses performance limitations by expressing them as deep hierarchical Gaussian filters (HGFs). Current PC networks suffer from slow training and performance degradation in deeper architectures due to fixing the precision matrix to identity. This novel method restores precision-weighted message passing, enabling dynamic uncertainty estimates and Hebbian-compatible update rules at every layer. The resulting networks can simultaneously learn activations, weights, and precisions using a single free-energy objective, eliminating the need for global error signals, iterations, or automatic differentiation. Benchmarked on FashionMNIST, this solution achieves epoch-level wall-clock costs comparable to backpropagation while converging in fewer epochs, and demonstrates superior performance in online, data efficiency, and concept-drift tasks. This work establishes closed-form variational inference with online precision learning as a viable foundation for deep PC networks, preserving their biological and interpretative benefits.

Key takeaway

For Machine Learning Engineers exploring alternatives to backpropagation, this research suggests you can achieve competitive performance with biologically plausible predictive coding. You should consider implementing closed-form variational inference with hierarchical Gaussian filters to improve training speed and depth scalability. This approach offers dynamic uncertainty estimates and superior performance in online learning, data efficiency, and concept-drift scenarios, potentially simplifying your model development for adaptive systems.

Key insights

Closed-form predictive coding via hierarchical Gaussian filters enables deep, efficient, and biologically plausible neural network training.

Principles

• Precision-weighted errors are crucial for deep predictive coding.
• Variational inference can unify learning of activations, weights, and precisions.
• Local, Hebbian-compatible rules can replace global error signals.

Method

Express predictive coding networks as deep hierarchical Gaussian filters (HGFs) to restore precision-weighted message passing. Learn activations, weights, and precisions under a single free-energy objective without iterations or automatic differentiation.

In practice

• Improve deep predictive coding network performance.
• Enhance online learning and data efficiency.
• Address concept-drift tasks effectively.

Topics

• Predictive Coding
• Hierarchical Gaussian Filters
• Variational Inference
• Neural Network Training
• Online Learning
• Concept Drift

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Predictive Coding with Bayesian Priors via Proximal Gradients — Predictive coding is reinterpreted as continuous-time proximal gradient descent on a regularized maximum-a-posteriori objective.
• Gaussian Process Prior Variational Autoencoder for Endoscopic Videos — A Gaussian Process Prior Variational Autoencoder (GPVAE) leverages temporal continuity for robust, uncertainty-aware endoscopic video restoration.
• Residual Modeling for High-Fidelity Learned Compression of Scientific Data — Tailored residual representations preserve learned compression advantages in high-fidelity scientific data scenarios.
• Towards Accurate and Calibrated Classification: Regularizing Cross-Entropy From A Generative Perspective — Generative Cross-Entropy improves DNN accuracy and calibration by regularizing class-level confidence from a generative perspective.
• Predictive variational inference: Learn the predictively optimal posterior distribution — PVI optimizes posterior predictive distributions for robustness against model misspecification, unlike traditional Bayesian methods.
• FFR: Forward-Forward Learning for Regression — FFR extends Forward-Forward learning to regression using ordinal competitive learning and a stratified architecture, achieving near-BP accuracy with less memory.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.