Universality of Shallow and Deep Neural Networks on Non-Euclidean Spaces

2026-03-02 · Source: cs.NE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

This paper introduces a framework for shallow and deep neural networks, termed Topological Feedforward Neural Networks (TFNNs), that operate on general topological spaces rather than being restricted to Euclidean inputs. The model uses a family of continuous feature maps and a fixed scalar activation function, reducing to standard feedforward networks in Euclidean cases. The core focus is on the universal approximation property (UAP), establishing conditions under which TFNNs are dense in spaces of continuous vector-valued functions on arbitrary and locally convex topological spaces. A significant contribution is the analysis of "deep narrow" TFNNs, where hidden layer width is bounded while depth can grow. The research identifies conditions for these width-constrained deep networks to retain UAP, extending classical approximation theorems. As a concrete example, it employs Ostrand's extension of the Kolmogorov superposition theorem to derive explicit universality results for products of compact metric spaces, with width bounds tied to topological dimension.

Key takeaway

For AI Scientists working with neural networks on non-Euclidean data, this research demonstrates that universal approximation is achievable. You should consider the topological properties of your input space, as the "D-property" or "finite-dimensional composition property" can guide network design. Specifically, for compact metric spaces, Ostrand's theorem provides a concrete path to deep narrow universality with explicit width bounds tied to topological dimension, offering a principled way to manage model complexity.

Key insights

TFNNs extend universal approximation to non-Euclidean spaces, even with width constraints, by leveraging topological properties.

Principles

• The D-property enables universality without width constraints.
• Finite-dimensional composition allows deep narrow universality.
• Topological dimension can dictate network width bounds.

Method

The method involves defining TFNNs with feature maps from a general topological space, then establishing UAP conditions based on a "D-property" for unconstrained networks and a "finite-dimensional composition property" for deep narrow architectures.

In practice

• Apply TFNNs to data from non-Euclidean domains.
• Use Ostrand inner functions for products of compact metric spaces.
• Consider topological dimension for deep narrow network design.

Topics

• Universal Approximation
• Topological Spaces
• Deep Narrow Networks
• Kolmogorov-Ostrand Theorem
• Topological Dimension

Best for: AI Scientist, AI Researcher, Research Scientist, AI Student

Related on AIssential

• Algorithmic Foundations of Deep Learning: Complexity-Theoretic Rates and a Characterization of Universal Approximation — Neural network complexity is governed by algorithmic complexity, not just regularity, viewing NNs as computational models.
• Neural Networks on Symmetric Spaces of Noncompact Type — A unified point-to-hyperplane distance enables novel neural network architectures on noncompact symmetric spaces.
• Dense Neural Networks are not Universal Approximators — Dense neural networks are not universal approximators under practical weight and dimension constraints.
• Weighted universal approximation of differentiable maps on infinite-dimensional manifolds — The universal approximation theorem for FNNs is generalized to differentiable maps, including derivative approximation, via a weighted Nachbin theorem.
• Sublinearly Structured Deep Neural Networks Achieve Feature Learning Consistency for Compositional Functions — Sublinearly structured DNNs achieve feature-learning consistency and universal approximation for compositional functions, even when over-parameterized.
• Pointwise Generalization in Deep Neural Networks — Pointwise Riemannian Dimension explains deep network generalization by characterizing feature representation complexity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.NE updates on arXiv.org.