Compositional Approximation Can Strictly Outperform Superpositional Approximation

2026-06-07 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, quick

Summary

A new theoretical study demonstrates that compositional approximation methods can strictly outperform superpositional approximation for specific function classes. While many classical function classes are optimally approximated by superpositional methods, achieving polynomial decay of uniform approximation error, this work constructs explicit examples where compositional methods yield arbitrarily larger gaps in approximation rates. The research highlights structural properties within function classes that inherently limit the performance of superpositional techniques, suggesting that methods like neural networks, which are compositional, can achieve superior approximation rates under certain conditions, even when parameter encoding constraints are considered. This challenges the assumption of comparable approximation rates between the two method types, published on 2026-06-07.

Key takeaway

For research scientists evaluating function approximation techniques, this work indicates that relying solely on superpositional methods may lead to suboptimal results for certain complex function classes. You should investigate the structural properties of the functions you are approximating, as compositional approaches, such as neural networks, might be essential to achieve the highest possible approximation rates. Consider exploring compositional architectures when traditional linear combination methods fall short, especially for problems requiring high precision.

Key insights

Compositional approximation can strictly outperform superpositional methods for certain function classes, achieving arbitrarily larger approximation rate gaps.

Principles

• Optimality in approximation implies polynomial decay of uniform error.
• Function class structure can limit superpositional approximation rates.

Topics

• Compositional Approximation
• Superpositional Approximation
• Function Approximation
• Approximation Theory
• Neural Networks
• Approximation Rates

Best for: AI Scientist, Research Scientist

Related on AIssential

• Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspective — Transformers can overcome the curse of dimensionality, demonstrating strong expressive capabilities for function approximation.
• Compositional Boundaries for Density Fusion — Order-invariant distributed density fusion requires specific algebraic properties, often violated by divergence-based or approximate methods.
• Generalization error bounds for two-layer neural networks with Lipschitz loss function — Generalization error bounds for two-layer neural networks can be derived without bounded loss functions using Wasserstein distance and SGM moment bounds.
• Efficient frequent directions algorithms for approximate decomposition of matrices and higher-order tensors — New frequent directions algorithms enhance low-rank matrix and tensor approximations for efficiency and accuracy.
• Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity — O-minimal definability ensures finite PAC sample complexity for a wide range of feedforward neural networks.
• BRo-JEPA: Learning Modular Arithmetic in Latent Space — Latent world models can learn symbolic rules if their architecture aligns with the problem's inherent structure.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.