Weighted universal approximation of differentiable maps on infinite-dimensional manifolds

2026-06-08 · Source: Machine Learning · Field: Science & Research — Mathematics & Computational Sciences, Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new generalization of the universal approximation theorem (UAT) has been developed for functional input neural networks (FNNs), extending their capability to approximate differentiable maps by also including the approximation of their derivatives. This advancement moves beyond the usual UAT formulation on compact sets. The FNN architecture maps inputs from a potentially infinite-dimensional weighted manifold to a real-valued hidden layer, where a non-linear scalar activation function is applied, before outputting to a Banach space via linear readouts. The core of this generalization relies on proving a weighted Nachbin theorem. Practical applications include approximating non-anticipative functionals, encompassing both horizontal and vertical derivatives, and demonstrating that linear functions of the signature can approximate path space functionals, including their directional derivatives.

Key takeaway

For AI Scientists and Research Scientists developing models for complex, infinite-dimensional data or path-dependent processes, this generalized universal approximation theorem implies that functional input neural networks (FNNs) can accurately model not only function values but also their derivatives. You should consider FNNs for applications where understanding rates of change, sensitivities, or directional derivatives is critical, such as in mathematical finance or control systems, offering robust approximation capabilities beyond traditional compact set limitations.

Key insights

The universal approximation theorem for FNNs is generalized to differentiable maps, including derivative approximation, via a weighted Nachbin theorem.

Principles

• FNNs can approximate differentiable maps and their derivatives.
• Approximation extends beyond compact sets to infinite dimensions.
• Weighted Nachbin theorem enables derivative approximation.

Method

Establish a universal approximation theorem for differentiable maps and their derivatives by proving a weighted Nachbin theorem, extending FNN capabilities beyond compact sets.

In practice

• Approximate non-anticipative functionals with derivatives.
• Use signature's linear functions for path space functionals.

Topics

• Functional Input Neural Networks
• Universal Approximation Theorem
• Differentiable Maps
• Infinite-Dimensional Manifolds
• Nachbin Theorem
• Path Space Functionals

Best for: AI Scientist, Research Scientist

Related on AIssential

• Universality of Shallow and Deep Neural Networks on Non-Euclidean Spaces — TFNNs extend universal approximation to non-Euclidean spaces, even with width constraints, by leveraging topological properties.
• Learning functional components of PDEs from data using neural networks — Neural networks can approximate unknown functional components within PDEs by embedding and training them with data.
• Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approcimations — Neural flow operators provide a unified framework proving universal approximation for both finite and infinite-dimensional spaces.
• Certified and accurate computation of function space norms of deep neural networks — Certified function space norms for neural networks are achieved via interval arithmetic and adaptive quadrature.
• Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspective — Transformers can overcome the curse of dimensionality, demonstrating strong expressive capabilities for function approximation.
• Neural Network Compression by Approximate Differential Equivalence — Compress neural networks by aggregating functionally similar neurons using differential equivalence, not just pruning weights.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.