GNet: A scalable and flexible Gaussian process network with nonparametric neurons

2026-07-14 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences, Data Science & Analytics · Depth: Expert, extended

Summary

GNet, a scalable and flexible Gaussian process network, employs nonparametric activation functions modeled by Gaussian processes. To address the high computational and storage costs typically associated with large datasets, GNet incorporates the jointly inverse Kalman filter (JIKF). This fast algorithm reduces the computational order for covariance matrix-vector multiplication from ℮(n²) to ℮(nq²), where "n" is the sample size and "q" is the latent state dimension (e.g., 1 for exponential kernel, 3 for Matérn kernel). Additionally, GNet utilizes closed-form expressions for loss gradients, avoiding large computational graphs from automatic differentiation, and integrates JIKF with a Nyström preconditioned conjugate gradient algorithm (JIKF-PCG) for predictions that scale logarithmically with sample size. Benchmarking shows GNet achieves competitive performance across diverse tasks, including nonlinear function prediction and high-dimensional input problems in classical density functional theory, often with significantly fewer parameters (e.g., 2,031 vs. 629,249 in one experiment) and reduced predictive error.

Key takeaway

For machine learning engineers or research scientists building scalable predictive models, GNet offers a compelling alternative to traditional Gaussian processes or neural networks. You should consider GNet for tasks requiring high predictive accuracy on large datasets, particularly when computational and storage costs are a concern. Its ability to handle high-dimensional inputs and complex nonlinearities with significantly fewer parameters can streamline model deployment and reduce infrastructure demands.

Key insights

GNet combines Gaussian processes and neural networks with a fast Kalman filter variant for scalable, accurate, and cost-effective predictive modeling.

Principles

• Nonparametric activations enhance model flexibility.
• Kalman filter variants accelerate GP computations.
• Closed-form gradients improve training scalability.

Method

GNet replaces parametric activation functions with Gaussian processes. It uses the jointly inverse Kalman filter (JIKF) for ℮(nq²) covariance matrix-vector multiplication and JIKF-PCG for logarithmic-cost predictions, trained via Adam with closed-form gradients.

In practice

• Predict nonlinear functions accurately.
• Perform nonparametric regression efficiently.
• Model high-dimensional physical theory inputs.

Topics

• Gaussian Process Networks
• Kalman Filter Algorithms
• Scalable Predictive Modeling
• Nonparametric Activation Functions
• Computational Efficiency
• High-Dimensional Regression

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• The Theory and Practice of Highly Scalable Gaussian Process Regression with Nearest Neighbours — NNGP/GPnn offer scalable, principled Gaussian process regression with minimax-optimal rates and hyperparameter robustness.
• Your GFlowNet Secretly Learns an Optimal Transport Plan — GFlowNets can learn optimal transport plans by reducing their objective to a Kantorovich OT problem.
• ParaRNN: Large-Scale Nonlinear RNNs, Trainable in Parallel — ParaRNN enables parallel training of large-scale nonlinear RNNs, matching transformer performance with superior inference efficiency.
• Do Newer Lightweight CNNs Perform Better Under Resource Constraints? A Controlled Multigenerational Study of Architecture, Initialization, Training Budget, and Efficiency — Lightweight CNNs demonstrate selective performance and efficiency gains, not universal improvements across diverse resource limitations.
• Semi-Supervised Learning on Graphs using Graph Neural Networks — GNN performance in semi-supervised learning is bounded by approximation, stochastic, and optimization errors, influenced by label scarcity and graph topology.
• GUIDED Network-Agnostic Feature Initialization for Spatial Transferability in GNN-based Models — GUIDED enables GNNs to achieve spatial transferability for traffic assignment by standardizing demand input across diverse network topologies.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.