Guaranteed Nonconvex Low-Rank Tensor Estimation via Scaled Gradient Descent

2025-12-31 · Source: JMLR · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

A new scaled gradient descent (ScaledGD) algorithm has been developed for low-rank tensor estimation, addressing challenges posed by corrupted multi-dimensional data in signal processing and machine learning. Published by Tong Wu in 2026, ScaledGD directly estimates tensor factors using spectral initializations within the tensor-tensor product (t-product) and tensor singular value decomposition (t-SVD) framework. The algorithm offers tailored variants for robust tensor principal component analysis, robust tensor completion, and tensor regression. Theoretically, ScaledGD achieves linear convergence at a constant rate, independent of the ground truth low-rank tensor's condition number, while maintaining low per-iteration costs. This marks the first algorithm with such proven properties for t-SVD-based low-rank tensor estimation, demonstrating efficacy in accelerating convergence for ill-conditioned problems.

Key takeaway

For research scientists working with corrupted multi-dimensional data and low-rank tensor estimation, adopting the ScaledGD algorithm can significantly improve computational efficiency. Its proven linear convergence rate, independent of the tensor's condition number, means you can achieve reliable results faster, especially with ill-conditioned data. Consider integrating ScaledGD into your tensor analysis workflows to accelerate convergence and enhance robustness in applications like tensor completion or regression.

Key insights

ScaledGD offers provably fast, condition-number-independent convergence for low-rank tensor estimation using t-SVD.

Principles

• Tensor factors can be directly estimated.
• Spectral initialization improves tensor estimation.
• Linear convergence is achievable for tensor problems.

Method

ScaledGD estimates tensor factors directly with tailored spectral initializations under the t-product and t-SVD framework, achieving linear convergence independent of the condition number.

In practice

• Apply ScaledGD to robust tensor PCA.
• Use ScaledGD for tensor completion tasks.
• Implement ScaledGD in tensor regression.

Topics

• Scaled Gradient Descent
• Low-Rank Tensor Estimation
• Tensor Singular Value Decomposition
• Tensor-Tensor Product
• Nonconvex Optimization

Code references

• twugithub/Tensor-ScaledGD

Best for: Research Scientist, AI Scientist

Related on AIssential

• Low-Rank Tensor Completion Based on Fractional Regularization with Ky Fan p-k Norm — A novel nonconvex surrogate, TNPK, improves low-rank tensor completion by accurately approximating tubal rank with flexible parameters and proven convergence.
• stochastic gradient descent #maths #statistics #datascience #machinelearning #dataanlysis — Faster iteration with noisy gradients often outperforms slow, precise convergence in machine learning.
• Natural gradient descent with momentum — Natural gradient descent with momentum improves optimization for nonlinear models by incorporating inertial dynamics.
• Efficient frequent directions algorithms for approximate decomposition of matrices and higher-order tensors — New frequent directions algorithms enhance low-rank matrix and tensor approximations for efficiency and accuracy.
• SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices — SigmaScale learns activation-aware scaling matrices to reduce effective intrinsic rank, improving SVD-based LLM compression performance.
• Functional Gradient Descent with Adaptive Representations — Adaptive representation in functional gradient descent ensures convergence to global minimizers by dynamically refining gradient approximations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by JMLR.