Nonlinear function-on-function regression by RKHS

2025-12-31 · Source: JMLR · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

Peijun Sang and Bing Li propose a novel nonlinear function-on-function regression model designed for scenarios where both the covariate and response are random functions. Their method involves a two-step process: initially constructing Hilbert spaces for the functional covariate and response, followed by building a second-layer Hilbert space for the covariate to capture nonlinearity. This second-layer space is a reproducing kernel Hilbert space (RKHS), generated by a positive definite kernel derived from the first-layer Hilbert space's inner product, forming a nested Hilbert space structure. The researchers developed estimation procedures that accommodate functional data observed at varying time points across subjects. They also established the convergence rate of their estimator and the weak convergence of the predicted response within the Hilbert space. Numerical studies, including simulations and a real data application, were conducted to evaluate the estimator's finite sample performance.

Key takeaway

For AI Scientists and Research Scientists working with functional data, this model offers a robust approach to nonlinear function-on-function regression. You should consider implementing this nested Hilbert space and RKHS framework when your covariates and responses are complex random functions, especially if your data is observed asynchronously across subjects, to achieve improved estimation and prediction accuracy.

Key insights

Nested Hilbert spaces and RKHS enable robust nonlinear function-on-function regression for complex functional data.

Principles

• Nonlinearity requires a second-layer Hilbert space.
• RKHS can capture complex functional relationships.

Method

Construct initial Hilbert spaces, then build a second-layer RKHS for the covariate using a kernel derived from the first-layer inner product, followed by estimation procedures.

In practice

• Analyze functional data with varying observation times.
• Apply to complex nonlinear functional relationships.

Topics

• Nonlinear Regression
• Function-on-Function Regression
• Reproducing Kernel Hilbert Space
• Functional Data Analysis
• Nested Hilbert Spaces

Code references

• JmlrOrg/v27

Best for: AI Scientist, Research Scientist

Related on AIssential

• Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence — A new theory for RKHS M-estimation provides sharp convergence rates and explains how these estimators avoid the curse of dimensionality.
• Generalized Functional ANOVA in Closed-Form: A Unified View of Additive Explanations — A new method provides a tractable, explicit functional ANOVA decomposition for dependent continuous inputs, enhancing model interpretability.
• Towards Unified Native Spaces in Kernel Methods — A new unified kernel class integrates diverse positive definite kernels, enabling characterization of associated Sobolev spaces.
• FunctionEvolve: Structure-Guided Symbolic Regression with LLMs — Structure-guided symbolic regression with LLMs significantly improves exact form recovery by using expression trees.
• Flexible Functional Treatment Effect Estimation — A WMKRR method directly minimizes uniform balancing error for functional treatment effect estimation.
• Probabilistic function-on-function nonlinear autoregressive model for emulation and reliability analysis of stochastic dynamical systems — F2NARX uses function-on-function autoregression with sparse Gaussian processes for efficient, probabilistic emulation of complex dynamical systems.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by JMLR.