Manifold Dimension Estimation via Local Graph Structure

2026-07-20 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

A new framework for manifold dimension estimation, integrating Principal Component Analysis (PCA) with regression techniques, is introduced to effectively capture a manifold's local graph structure. This approach explicitly models underlying curvature, moving beyond the traditional flatness assumption. Two representative estimators, Quadratic Embedding (QE) and Total Least Squares (TLS), are detailed. QE uses ordinary least squares with an F-statistic for model evaluation, while TLS employs total least squares, assessing fit via relative drops in total error. Experiments on 18 synthetic datasets (with n=500 and n=2000 samples, both noiseless and noisy) and 6 additional nonlinearly embedded manifolds demonstrate that QE and TLS perform competitively with, and often outperform, state-of-the-art methods like Local PCA, CA-PCA, TwoNN, and DanCo. Their advantages are particularly evident with small sample sizes, nonlinear embeddings, and noisy data. Real-world evaluations on ISOMAP (698 images, 3D), MNIST (6,742 images, 8-11D), and ISOLET (6,240 vectors, 16-22D) datasets further confirm their robustness and accuracy, especially where ambient dimensions significantly exceed intrinsic dimensions.

Key takeaway

For research scientists or AI practitioners working with high-dimensional data, consider Quadratic Embedding (QE) or Total Least Squares (TLS) for intrinsic dimension estimation. These methods excel with noisy data, small sample sizes, and nonlinearly embedded manifolds, offering superior accuracy over traditional flatness-assumption techniques. You should prioritize these curvature-aware approaches when existing estimators struggle with complex data geometries or high ambient dimensions.

Key insights

Integrating PCA with regression captures local graph structure for robust manifold dimension estimation.

Principles

• Explicitly model manifold curvature via local graph functions.
• Regression-based fitting enhances estimation stability.
• Robustness to noise and varying neighborhood sizes is crucial.

Method

The framework applies local PCA to neighborhoods, then fits quadratic regression models (OLS for QE, TLS for TLS) to PCA coordinates, evaluating fit via F-statistics or relative error drops to determine intrinsic dimension.

In practice

• Use QE/TLS for noisy, nonlinearly embedded data.
• Prioritize curvature-aware methods for complex manifolds.
• Consider QE for smaller sample sizes.

Topics

• Manifold Dimension Estimation
• Local Graph Structure
• Principal Component Analysis
• Quadratic Embedding
• Total Least Squares
• Nonlinear Embeddings

Code references

• loong-bi/manifold-dimension-estimation

Best for: AI Scientist, Research Scientist

Related on AIssential

• The Data Don’t Fill the Space — Manifold geometry analyzes data concentrated on a low-dimensional, curved surface embedded in a higher-dimensional space.
• Generalization in Nonlinear Least Squares via Learned Feature Geometry — New generalization bounds for nonlinear least-squares models depend on learned feature geometry and data-dependent effective dimension, not just parameter count.
• Towards an Asymptotic Efficiency Theory on Regular Parameter Manifolds — A unified asymptotic efficiency theory extends classical statistical optimality concepts to Riemannian manifold-valued parameters.
• Geometry-Aware Bayesian Quantification via Compositional Data Analysis — A geometry-aware KDE model improves multiclass quantification by respecting the probability simplex.
• Constructing VAE Latent Spaces with Prescribed Topology — A new VAE framework aligns latent space topology with data manifold topology, improving representation and reconstruction.
• Rethinking Structural Anomaly Detection: From Decision Boundaries to Projection Operators — Structural anomaly detection improves by projecting data onto a normal manifold, identifying anomalies via projection residuals.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.