Efficient frequent directions algorithms for approximate decomposition of matrices and higher-order tensors

2025-12-31 · Source: JMLR · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

Maolin Che, Yimin Wei, and Hong Yan introduce efficient frequent directions (FD) algorithms for low-rank matrix and tensor approximations. The research develops two algorithms for matrix approximations using embedding matrices formed by products of sparse embedding (SpEmb) matrices with either standard Gaussian or subsampled randomized Hadamard transform (SRHT) matrices. Theoretical bounds for singular values of these matrices support the algorithms. The authors also present several FD-based randomized variants of T-HOSVD and ST-HOSVD for approximate Tucker decomposition, given a specific Tucker-rank. Additionally, efficient FD-based randomized algorithms are proposed for approximate tensor-train (TT) decomposition with a given TT-rank. The study validates the efficiency and accuracy of these new algorithms using both synthetic and real-world matrix and tensor datasets.

Key takeaway

For research scientists working with large datasets requiring low-rank approximations, these new FD-based algorithms offer improved efficiency and accuracy for matrix, Tucker, and tensor-train decompositions. You should consider integrating these methods, particularly when dealing with embedding matrices composed of sparse embedding and Gaussian or SRHT matrices, to optimize computational performance and result precision in your models.

Key insights

New frequent directions algorithms enhance low-rank matrix and tensor approximations for efficiency and accuracy.

Principles

• Combine sparse embedding with Gaussian/SRHT matrices.
• Leverage singular value bounds for theoretical guarantees.

Method

Develops FD-based randomized variants for T-HOSVD, ST-HOSVD (Tucker decomposition), and algorithms for TT decomposition, using specific embedding matrix compositions.

In practice

• Apply to low-rank matrix approximation.
• Compute approximate Tucker decomposition.
• Perform approximate tensor-train decomposition.

Topics

• Frequent Directions Algorithm
• Low-rank Matrix Approximation
• Tucker Decomposition
• Tensor-Train Decomposition
• Sparse Embedding

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Efficient Online Prediction for High-Dimensional Time Series via Joint Tensor Tucker Decomposition — TOPA-AAW offers efficient, accurate online prediction for high-dimensional streaming tensor time series via adaptive tensor factorization.
• An Asymptotically Optimal Coordinate Descent Algorithm for Learning Bayesian Networks from Gaussian Models — A new coordinate descent algorithm offers provable optimality guarantees for learning Bayesian networks from Gaussian data.
• Thresholded Local Hyper-Flow Diffusion — TL-HFD offers an exact, localized first-order method for seeded submodular hypergraph clustering, improving efficiency and noise robustness.
• Low-Rank Tensor Completion Based on Fractional Regularization with Ky Fan p-k Norm — A novel nonconvex surrogate, TNPK, improves low-rank tensor completion by accurately approximating tubal rank with flexible parameters and proven convergence.
• Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing — Identifies degenerate Gaussian mixture latent variables from piecewise affine observations using sparsity regularization for disentanglement.
• A Typed Tensor Language for Federated Learning — The language formalizes federated learning computations, enabling fixed-dimensional shared state communication for efficiency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by JMLR.