Unsupervised Feature Selection via Nonnegative Orthogonal Constrained Regularized Minimization

2025-12-31 · Source: JMLR · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Yan Li, Defeng Sun, and Liping Zhang introduced a new unsupervised feature selection model called Nonnegative Orthogonal Constrained Regularized Minimization (NOCRM). This model integrates feature selection with nonnegative spectral clustering and includes mechanisms to prevent overfitting. To address the NOCRM model, the researchers developed an inexact augmented Lagrangian multiplier method, utilizing a proximal alternating minimization approach for its subproblems. They provide a rigorous theoretical proof demonstrating that the algorithm's sequence converges to a stationary point of the model. Numerical experiments conducted on various datasets indicate that NOCRM exhibits stability and robustness, outperforming several existing methods in clustering evaluation metrics. The code for NOCRM is publicly available on GitHub.

Key takeaway

For research scientists developing or applying unsupervised feature selection techniques, NOCRM provides a robust, theoretically-backed alternative. Its proven convergence guarantees and superior performance in clustering metrics suggest it can enhance model stability and accuracy. Consider integrating NOCRM into your feature engineering pipeline, especially for datasets where overfitting is a concern, and evaluate its impact on downstream clustering tasks.

Key insights

NOCRM offers a theoretically sound unsupervised feature selection method with convergence guarantees.

Principles

• Embed feature selection into nonnegative spectral clustering.
• Nonnegative orthogonality constraints prevent overfitting.

Method

An inexact augmented Lagrangian multiplier method solves NOCRM, with subproblems handled by proximal alternating minimization.

In practice

• Apply NOCRM for dimensionality reduction in big data.
• Use NOCRM to improve clustering evaluation metrics.

Topics

• Unsupervised Feature Selection
• Nonnegative Orthogonal Constraints
• Augmented Lagrangian Method
• Spectral Clustering
• Convergence Guarantees

Code references

• liyan-amss/NOCRM_code
• JmlrOrg/v27

Best for: Research Scientist, AI Researcher, AI Scientist, AI Student

Related on AIssential

• On solving symmetric multi-type orthogonal non-negative matrix tri-factorization problem — The paper introduces two robust heuristic algorithms for symmetric multi-type orthogonal non-negative matrix tri-factorization, improving network analysis and clustering.
• Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization — LLM-driven kernel discovery via a two-stage process and LOO-CRPS significantly improves high-dimensional Bayesian optimization.
• A Data-Informed Variational Clustering Framework for Noisy High-Dimensional Data — DIVI is a variational clustering framework that jointly learns feature relevance and adapts cluster structure in noisy high-dimensional data.
• FOSC-X: An Extended Framework for Optimal Local Cuts and Non-Horizontal Cluster Selection from Clustering Hierarchies — FOSC-X extends hierarchical clustering to extract top-M optimal flat solutions with flexible cluster-count constraints via dynamic programming.
• An unsupervised decision-support framework for multivariate biomarker analysis in athlete monitoring — Unsupervised multivariate analysis can identify latent physiological states and silent risk in athlete monitoring.
• Why Not Hyperparameter-Friendly Optimisation? A Monotonic Adaptive Norm Rescaling Approach For Long-Tailed Recognition — SAMN improves long-tailed recognition by enforcing monotonic per-class weight norms without sensitive hyperparameters.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by JMLR.