Generalized Least Squares Kernelized Tensor Factorization

2026-07-22 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

The Generalized Least Squares Kernelized Tensor Factorization (GLSKF) framework addresses the challenge of completing multidimensional tensor data with missing values. It integrates a smoothness-constrained low-rank factorization, which captures global dependencies, with a locally correlated residual process to effectively model short-scale, high-frequency variations. GLSKF defines a covariance norm to enforce factor matrix smoothness and employs structured covariance functions for local processes. Its model estimation uses an Alternating Least Squares procedure with closed-form solutions, leveraging projection matrices that preserve Kronecker structures for efficient computation via Conjugate Gradient and Preconditioned Conjugate Gradient algorithms. This approach achieves a computational cost of O(J_CG/PCG * R * N), scaling linearly with data size. Evaluated on traffic speed imputation, color image inpainting, video completion, and MRI image reconstruction, GLSKF demonstrates superior effectiveness and scalability.

Key takeaway

For Machine Learning Engineers or Research Scientists tackling incomplete multidimensional data, GLSKF provides a highly effective tensor completion solution. You should consider implementing GLSKF when your datasets exhibit both broad global patterns and intricate local variations, as it uniquely captures both. Its linear computational cost and ability to operate with smaller tensor ranks make it particularly efficient for large-scale applications like image inpainting or traffic imputation, offering superior accuracy over traditional low-rank methods.

Key insights

GLSKF additively integrates global low-rank factorization and local correlated residuals for scalable, accurate multidimensional tensor completion.

Principles

• Additive models effectively capture global and local data structures.
• Covariance norms offer a generalized framework for smoothness constraints.
• Kronecker products and sparse matrices enable scalable tensor operations.

Method

GLSKF employs an ALS procedure, solving subproblems via CG/PCG algorithms, leveraging Kronecker matrix-vector multiplication (MVM) and sparse matrix computations for efficiency.

In practice

• Apply GLSKF for traffic, image, video, and MRI data completion.
• Use sparse covariance/precision matrices for local component efficiency.
• Employ smaller tensor CP ranks (e.g., R=10-20) for global factors.

Topics

• Tensor Completion
• Kernelized Factorization
• Generalized Least Squares
• Multidimensional Data
• Sparse Matrix Computations
• Conjugate Gradient

Code references

• MengyingLei/GLSKF
• zhiyongc/Seattle-Loop-Data
• liyaguang/DCRNN

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Graph-Regularized Low-Rank Matrix Completion by Variable Projection — Graph regularization enhances low-rank matrix completion by leveraging row/column relationships for improved accuracy and robustness.
• Guaranteed Nonconvex Low-Rank Tensor Estimation via Scaled Gradient Descent — ScaledGD offers provably fast, condition-number-independent convergence for low-rank tensor estimation using t-SVD.
• Kernel Regression with Tensor Trains and Hadamard Overparameterization — KReTTaH integrates kernel regression, tensor trains, and Hadamard overparameterization via Riemannian optimization for efficient, interpretable multi-way data imputation.
• Efficient frequent directions algorithms for approximate decomposition of matrices and higher-order tensors — New frequent directions algorithms enhance low-rank matrix and tensor approximations for efficiency and accuracy.
• Low-Rank Tensor Completion Based on Fractional Regularization with Ky Fan p-k Norm — A novel nonconvex surrogate, TNPK, improves low-rank tensor completion by accurately approximating tubal rank with flexible parameters and proven convergence.
• Geometric Gradient Rectification for Safe Open-Set Semi-Supervised Learning — GGR rectifies conflicting auxiliary gradients in OSSL by projecting them onto an admissible region, improving robustness and generalization.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.