(MPO)$^2$: Multivariate Polynomial Optimization based on Matrix Product Operators

2026-07-17 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

The new framework, Multivariate Polynomial Optimization based on Matrix Product Operators, or (MPO)$^2$, addresses the challenge of exponentially growing coefficient tensors in multivariate polynomial models used for universal function approximation in machine learning and signal processing. While existing tensorized polynomial models like canonical polyadic decompositions and tensor train formulations reduce cost, they suffer from rank-limited expressivity and feature order dependency, respectively. (MPO)$^2$ overcomes these limitations by integrating learned Matrix Product Operator (MPO) feature embeddings with compact polynomial weight tensors. This approach results in feature order independent polynomial representations and supports the incorporation of structured operators such as projections, convolutions, and masks for weight tensor symmetries. Benchmarking across regression and classification tasks demonstrates that (MPO)$^2$ outperforms current tensor decomposition-based polynomial models, establishing itself as a flexible and efficient alternative for polynomial function approximation.

Key takeaway

For Machine Learning Engineers developing models for universal function approximation, if you are encountering scalability issues with traditional multivariate polynomial models due to exponential tensor growth, consider evaluating (MPO)$^2$. This framework offers a more efficient and flexible alternative by providing feature order independent representations and supporting structured operators. You should explore its application in your regression and classification tasks to potentially achieve improved performance and reduced computational overhead compared to existing tensor decomposition methods.

Key insights

(MPO)$^2$ offers a feature-order-independent, compact polynomial approximation by combining MPO embeddings with structured weight tensors, outperforming prior tensor methods.

Principles

• Multivariate polynomial models face exponential tensor growth.
• Tensor train methods are feature order dependent.
• MPO embeddings enable compact, order-independent representations.

Method

The framework combines learned MPO feature embeddings with compact polynomial weight tensors to create feature order independent polynomial representations, allowing for structured operators like projections and convolutions.

In practice

• Efficient polynomial function approximation.
• Improved regression and classification benchmarks.
• Incorporate structured operators (e.g., convolutions).

Topics

• Multivariate Polynomials
• Matrix Product Operators
• Tensor Decomposition
• Function Approximation
• Machine Learning
• Signal Processing

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Efficient frequent directions algorithms for approximate decomposition of matrices and higher-order tensors — New frequent directions algorithms enhance low-rank matrix and tensor approximations for efficiency and accuracy.
• Isotropic Fourier Neural Operators — Isotropic Fourier Neural Operators improve FNOs by enforcing spatial symmetries, boosting performance and reducing parameters.
• Gaussian Mixture Model with unknown diagonal covariances via continuous sparse regularization — BLASSO is extended to GMMs with unknown diagonal covariances, offering simultaneous parameter and component number estimation.
• STDE++: Polynomial-Time Amortization for Linear Differential Operators — STDE++ enables polynomial-time amortization for linear differential operators in neural networks.
• Quantifying and Optimizing Simplicity via Polynomial Representations — Polynomial representations offer a quantitative simplicity metric and regularizer for deep network generalization.
• Heat and Mat\'ern Kernels on Matchings — Geometric kernels for matchings can be constructed by respecting symmetries and incorporating smoothness, despite computational challenges.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.