A data-driven Fourier-mixture neural-network method for density estimation

2026-05-19 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

Duy-Minh Dang and Volter Entoma propose a data-driven Fourier-trained neural-network method for estimating fixed-horizon probability densities from empirical characteristic-function (CF) information. This estimator utilizes a positive Gaussian–Laplace mixture with a closed-form CF, enabling direct training in Fourier space while preserving nonnegativity and unit mass. The method addresses two sampling scenarios: direct i.i.d. sampling using an empirical CF from i.i.d. samples, and resampling-based pseudo-sampling for dependent data using an empirical pseudo-CF. The authors derive an expected $L_{2}$ error bound for the i.i.d. case, detailing contributions from Fourier truncation, empirical training error, discretization, and CF sampling error. For pseudo-sampling, a conditional analogue includes two additional pseudo-law discrepancy terms. Numerical experiments demonstrate competitive performance against Expectation–Maximization on Gaussian-mixture benchmarks, significant improvements on heavy-tailed targets like the Cauchy distribution, and effective estimation of one-year Australian equity return laws from resampled dependent data, with the framework extended to multiple dimensions.

Key takeaway

For AI Scientists and Data Scientists working on probability density estimation, this Fourier-trained neural network method offers a robust alternative to traditional physical-space estimators like EM. Its ability to directly train in Fourier space while preserving density properties, coupled with rigorous error bounds for both i.i.d. and dependent data, means you can achieve superior accuracy on non-Gaussian and heavy-tailed distributions. Consider integrating this approach, especially when dealing with complex, real-world time-series data requiring fixed-horizon law estimation, to potentially improve model fit and computational efficiency by decoupling from sample size $M$ during iterative training.

Key insights

A Fourier-trained neural network using Gaussian-Laplace mixtures estimates probability densities from empirical characteristic functions.

Principles

• Preserve nonnegativity and unit mass during Fourier-space training.
• Decompose $L_{2}$ error into distinct Fourier-domain contributions.
• Handle dependent data via resampling-based pseudo-sampling.

Method

The method trains a single-hidden-layer feedforward neural network to parameterize a positive Gaussian–Laplace mixture, optimizing a Fourier-domain loss function against empirical characteristic function data, with a two-stage optimization strategy.

In practice

• Apply affine preprocessing for highly oscillatory empirical CFs.
• Use Gaussian-Laplace mixtures for heavy-tailed or sharp-structured targets.
• Employ block bootstrap for dependent financial time series.

Topics

• Density Estimation
• Fourier-trained Neural Networks
• Empirical Characteristic Function
• Gaussian–Laplace Mixture Models
• L2 Error Bounds

Best for: AI Scientist, Research Scientist, Data Scientist

Related on AIssential

• Density Estimation with Gaussian Mixture Models (GMM) and Empirical Priors — Density estimation approximates complex probability distributions using simpler component functions when closed-form solutions are unavailable.
• fTNN: a tensor neural network for fractional PDEs — fTNN is a deterministic tensor neural network method for fractional PDEs, using geometry-adapted integration and singularity-aware trial functions for high accuracy.
• Nonparametric Estimation of a Factorizable Density using Diffusion Models — Diffusion models can implicitly estimate factorizable densities, achieving minimax optimal rates.
• Regular Fourier Features for Nonstationary Gaussian Processes — The method efficiently approximates nonstationary Gaussian processes by discretizing spectral representation, avoiding probability assumptions.
• Deep regression learning from dependent observations with minimum error entropy principle — MEE-based deep neural networks achieve minimax optimal rates for nonparametric regression with dependent data.
• Sequential Fitting: A Different Perspective on the Spectral Bias of Neural Networks — The spectral bias in MLPs stems from "sequential fitting" and "boundary effects," not solely frequency content.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.