Regular Fourier Features for Nonstationary Gaussian Processes

2026-06-18 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, medium

Summary

Arsalan Jawaid and Jörg Seewig propose "Regular Fourier Features for Nonstationary Gaussian Processes" to address the Ó(n³) computational cost of simulating Gaussian processes (GPs). While existing Random Fourier Features (RFFs) reduce this to Ó(nm²) for stationary GPs, they are restrictive for nonstationary cases due to assumptions about spectral densities being probability measures. Their new method discretizes the spectral representation on a regular grid, preserving the correlation structure among spectral weights without requiring probability interpretations or symmetrization. This approach extends to harmonizable processes, including those with complex-valued spectral densities, and yields a low-rank approximation that is positive semi-definite by construction. The framework also supports kernel learning from data, even when the spectral density is unknown, through a factorized spectral parametrization. The method is demonstrated on locally stationary and harmonizable mixture kernels.

Key takeaway

For Machine Learning Engineers developing Gaussian Process models, this method offers a way to overcome the Ó(n³) computational bottleneck for nonstationary data. You can now efficiently simulate and learn complex nonstationary kernels, including those with complex-valued spectral densities, without restrictive probability assumptions. Consider implementing regular Fourier features to improve scalability and model flexibility in your nonstationary GP applications.

Key insights

The method efficiently approximates nonstationary Gaussian processes by discretizing spectral representation, avoiding probability assumptions.

Principles

• Spectral methods can reduce GP simulation complexity.
• Nonstationary GP spectral densities are not always probability measures.
• Preserving spectral weight correlation ensures positive semi-definiteness.

Method

Discretize the spectral representation on an equidistant frequency grid. Approximate the stochastic process as a sum of complex exponentials weighted by correlated spectral increments, then factorize the spectral matrix for a low-rank kernel approximation.

In practice

• Use for efficient nonstationary GP simulation.
• Apply to kernel learning with unknown spectral densities.
• Handle complex-valued spectral densities in harmonizable kernels.

Topics

• Gaussian Processes
• Nonstationary Kernels
• Fourier Features
• Spectral Methods
• Kernel Learning
• Computational Efficiency

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Generative Modeling of Approximately Periodic Time Series by a Posterior-Weighted Gaussian Process — A two-stage GP model generates stable, approximately periodic time series by decoupling intra- and inter-repetition variability.
• A data-driven Fourier-mixture neural-network method for density estimation — A Fourier-trained neural network using Gaussian-Laplace mixtures estimates probability densities from empirical characteristic functions.
• Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization — LLM-driven kernel discovery via a two-stage process and LOO-CRPS significantly improves high-dimensional Bayesian optimization.
• JA-SIREN: Deterministic Initialization for Sinusoidal Networks via Spectral Matching — Deterministic initialization via spectral matching for sinusoidal networks significantly enhances INR reproducibility and performance.
• Fourier fractal dimension to predict the generalization of deep neural networks — Fourier fractal dimension of network weight variations predicts deep neural network generalization without hold-out data.
• Federated Gaussian Process Learning via Pseudo-Representations for Large-Scale Multi-Robot Systems — pxpGP enables scalable, privacy-preserving Gaussian Process learning for large multi-robot systems via pseudo-representations and optimized ADMM.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.