Predictive Coding with Bayesian Priors via Proximal Gradients

2026-06-06 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new theoretical framework recasts predictive coding as continuous-time proximal gradient descent applied to a regularized maximum-a-posteriori (MAP) objective. For single-level problems, this approach demonstrates that proximal gradient descent precisely describes a leaky firing-rate network, where elements like membrane leak, recurrent matrix, and synaptic drive derive from a single optimization principle, aligning with Rao and Ballard's circuit. The prior determines the network's nonlinearity via its proximal operator, while likelihood precision sets observation gain. For multi-level hierarchies, a classical variable-splitting relaxation of the deep MAP problem yields hierarchical predictive coding, connecting local and distributed solvers by transforming the directed generative chain into an undirected Markov random field.

Key takeaway

For AI scientists researching biologically plausible learning or neural network dynamics, this framework offers a unified optimization perspective on predictive coding. You should consider how this reinterpretation, linking network components to a MAP objective via proximal gradients, could inform the design or analysis of hierarchical generative models, potentially simplifying their theoretical underpinnings and implementation.

Key insights

Predictive coding is reinterpreted as continuous-time proximal gradient descent on a regularized maximum-a-posteriori objective.

Principles

• Network components derive from one optimization principle.
• Prior selects nonlinearity via its proximal operator.
• Likelihood precision sets observation gain.

Method

Recasts predictive coding as continuous-time proximal gradient descent on a regularized MAP objective, employing variable-splitting relaxation for hierarchical structures.

Topics

• Predictive Coding
• Bayesian Priors
• Proximal Gradient Descent
• MAP Objective
• Neural Networks
• Markov Random Fields

Best for: Research Scientist, AI Scientist

Related on AIssential

• Closed-form predictive coding via hierarchical Gaussian filters — Closed-form predictive coding via hierarchical Gaussian filters enables deep, efficient, and biologically plausible neural network training.
• An Asymptotically Optimal Coordinate Descent Algorithm for Learning Bayesian Networks from Gaussian Models — A new coordinate descent algorithm offers provable optimality guarantees for learning Bayesian networks from Gaussian data.
• Response time of lateral predictive coding and benefits of modular structures — LPC networks can achieve minimal response times and modularity without sacrificing energetic efficiency or feature detection.
• Mitigating Forgetting in Continual Learning with Selective Gradient Projection — SFAO uses similarity-gated gradient updates to balance plasticity and stability in continual learning, reducing forgetting and memory.
• In-Context Learning for Latent Space Bayesian Optimization — Adapting tabular foundation models for latent space Bayesian optimization requires specific pretraining to address domain mismatches.
• Neural Network Compression by Approximate Differential Equivalence — Compress neural networks by aggregating functionally similar neurons using differential equivalence, not just pruning weights.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.