Adaptive generative moment matching networks for improved learning of dependence structures

2025-08-29 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

Adaptive Generative Moment Matching Networks (AGMMNs) are introduced, featuring an adaptive bandwidth selection procedure for the mixture kernel in the maximum mean discrepancy (MMD) to improve learning of dependence structures and copula random number generators. This method increases the number of kernels during training based on the relative error of the training loss, using validation loss for early stopping. AGMMNs maintain similar training times while significantly boosting training performance, as evidenced by validation MMD trajectories and values. Their superiority over standard GMMNs and parametric copula models is demonstrated across three applications: investigating convergence rates for estimators from high-dimensional copulas (up to 100 dimensions), improving training for a copula model derived from 50 S&P 500 constituents, and enhancing model prediction using both S&P 500 and FTSE 100 datasets.

Key takeaway

For quantitative analysts or machine learning engineers developing models for complex dependence structures, particularly in high-dimensional financial applications, you should consider implementing Adaptive Generative Moment Matching Networks (AGMMNs). This method offers significantly improved training performance and model prediction over traditional GMMNs and parametric copula models, without increasing training time, leading to more accurate simulations and forecasts.

Key insights

Adaptive bandwidth selection in GMMNs significantly improves learning of complex dependence structures and model prediction.

Principles

• Increase kernels during training based on training loss relative error.
• Utilize validation loss relative error as an early stopping criterion.

Method

An adaptive bandwidth selection procedure for the mixture kernel in MMD, increasing kernels during training based on the relative error of the training loss, with validation loss as an early stopping criterion.

In practice

• Model high-dimensional copula dependence structures.
• Improve prediction for financial market constituent data.
• Enhance Monte Carlo and quasi-Monte Carlo applications.

Topics

• Generative Moment Matching Networks
• Adaptive Bandwidth Selection
• Maximum Mean Discrepancy
• Copula Models
• Dependence Structures
• Financial Modeling
• Machine Learning

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• Generative Modeling of Approximately Periodic Time Series by a Posterior-Weighted Gaussian Process — A two-stage GP model generates stable, approximately periodic time series by decoupling intra- and inter-repetition variability.
• Minibatch Selection via Partition Matroid Constrained Gradient Matching — PartitionSel optimizes LLM fine-tuning on heterogeneous data by balancing domain coverage and convergence via gradient-matching and matroid constraints.
• Deep regression learning from dependent observations with minimum error entropy principle — MEE-based deep neural networks achieve minimax optimal rates for nonparametric regression with dependent data.
• ADAPTive Input Training for Many-to-One Pre-Training on Time-Series Classification — ADAPT enables mixed-batch pre-training for time-series data by aligning physical properties across diverse datasets.
• Finding Sparse Subnetworks in One Training Cycle via Progressive Magnitude-Based Pruning — Progressive magnitude-based pruning enables effective neural network sparsification within a single training cycle.
• JA-SIREN: Deterministic Initialization for Sinusoidal Networks via Spectral Matching — Deterministic initialization via spectral matching for sinusoidal networks significantly enhances INR reproducibility and performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.