The Data Don’t Fill the Space

2026-01-11 · Source: Agus’s Substack · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, long

Summary

Manifold geometry provides a framework for analyzing datasets that are smooth, continuous, and curved, existing within a lower-dimensional surface embedded in a higher-dimensional ambient space. This approach contrasts with coordinate geometry (flat spaces), similarity geometry (pairwise relations), and partition geometry (divisions). The core concept is the manifold hypothesis: data lie near a smooth set with an intrinsic dimension (d) much smaller than its ambient dimension (D). Unlike PCA, which only handles flat lower-dimensional structures, manifold methods like Isomap, LLE, Laplacian Eigenmaps, Diffusion Maps, and UMAP preserve intrinsic geometric relationships by approximating local linearity and stitching these approximations into a global picture. The `geomlearn` Python library, specifically `geomlearn.ch06_manifolds`, offers tools for building neighborhood graphs, performing Laplacian eigenmaps, and estimating intrinsic dimensionality, as demonstrated with a Swiss roll dataset.

Key takeaway

For Machine Learning Engineers working with high-dimensional data, understanding manifold geometry is crucial when your data exhibits non-linear, curved structures. Your traditional linear dimensionality reduction techniques like PCA will likely fail to preserve intrinsic relationships. You should apply manifold learning algorithms and diagnostics, such as those in `geomlearn`, to accurately model the data's true underlying geometry, ensuring more faithful and efficient representations.

Key insights

Manifold geometry analyzes data concentrated on a low-dimensional, curved surface embedded in a higher-dimensional space.

Principles

• Data often reside on a low-dimensional manifold.
• Manifolds are locally flat, globally curved.
• Euclidean distance can mislead on curved data.

Method

Manifold learning constructs a neighborhood graph from local adjacencies, then uses algorithms like Laplacian Eigenmaps to unfold the curved structure into a lower-dimensional embedding, preserving intrinsic geometric relationships.

In practice

• Use `geomlearn.ch06_manifolds` for manifold analysis.
• Check geodesic-to-Euclidean ratio for curvature.
• Verify spectral gap for intrinsic dimension.

Topics

• Manifold Geometry
• Intrinsic Dimension
• Manifold Hypothesis
• Neighborhood Graphs
• Laplacian Eigenmaps

Code references

• asudjianto-xml/geomlearn

Best for: AI Scientist, Machine Learning Engineer, Data Scientist

Related on AIssential

• The Data Manifold under the Microscope — A new framework provides ground-truth geometric data for validating deep learning theory and estimators.
• Learning the Geometry of Data: A Mathematical Review of Shape Space Analysis — Shape space analysis offers a unified geometric framework to extract patterns from complex data, outperforming traditional ML on subtle variations and sparse labels.
• Orthogonality and Dimensionality in Airline Cluster Analysis using PCA and Kernel PCA — Airline profit cycle clustering is geometrically robust across dimensionality reductions, but structural support for cluster count varies.
• Representation Gap: Explaining the Unreasonable Effectiveness of Neural Networks from a Geometric Perspective — Neural network generalization is geometrically characterized by the Representation Gap, scaling with intrinsic data dimension and model equivariance.
• You Have Been Doing Geometry All Along — Data science workflows are fundamentally geometric, shaping how models perceive and learn structure.
• Prerequisites II: Topology - Cristian Bodnar — Topology qualitatively defines proximity and continuity in spaces, abstracting beyond specific distance metrics.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Agus’s Substack.