The Geometry Underneath the Algebra

2026-01-11 · Source: Agus’s Substack · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mathematics & Computational Sciences · Depth: Advanced, long

Summary

This post elucidates the fundamental geometric concepts underpinning machine learning and data science, moving beyond algebraic formalities to explain how vectors, norms, inner products, projections, and linear maps describe size, direction, similarity, simplification, and transformation. It clarifies that vectors represent displacements, not just lists of numbers, and that choosing a norm (e.g., L1, L2, L∞) is a critical modeling decision defining "near" and "far." The inner product is presented as a measure of directional orientation, distinct from distance, while projection is explained as a controlled simplification process, exemplified by ordinary least squares regression and Principal Component Analysis (PCA). Furthermore, matrices are described as geometric operators that reshape space, and the covariance matrix is shown to define the shape and orientation of data clouds. The article concludes by detailing how eigenvectors and singular values reveal preferred directions and intrinsic dimensionality, with practical code examples using the `geomlearn` library for SVD analysis, subspace projection, and representation diagnostics.

Key takeaway

For Data Scientists and Machine Learning Engineers seeking to deepen their understanding of model behavior, grasping the geometric interpretations of core linear algebra concepts is crucial. This perspective clarifies why certain algorithms work and how modeling decisions, like norm selection, fundamentally alter outcomes. You should actively interpret mathematical objects like vectors, norms, and matrices not just as algebraic constructs, but as tools that define and transform the geometric properties of your data, leading to more informed algorithm design and debugging.

Key insights

A small set of geometric ideas forms the structural foundation of classical machine learning and statistics.

Principles

• Vectors encode displacement, not just position.
• Norm choice defines geometric distance and method behavior.
• Matrices are geometric actions, not just number arrays.

Method

The `geomlearn` library provides tools for SVD analysis, subspace projection, and representation diagnostics to understand data geometry and identify ill-conditioned representations.

In practice

• Use L1 norm for sparsity in regularization.
• Employ cosine similarity for direction-based comparisons.
• Diagnose representation health with condition number and effective rank.

Topics

• Geometric Machine Learning
• Vectors and Coordinate Systems
• Norms and Distance Metrics
• Inner Products
• Geometric Projection

Code references

• asudjianto-xml/geomlearn

Best for: Machine Learning Engineer, Data Scientist, AI Scientist

Related on AIssential

• Every Norm Draws a Circle — Norms provide distinct, valid methods for measuring vector size, each defining a unique geometric space.
• The Geometry of Linear Equations and Linear Regression — Geometric understanding of linear algebra clarifies solutions and underpins linear regression.
• How Linear Algebra Powers Machine Learning (ML) — Linear algebra underpins machine learning by transforming raw data into computable numerical structures and enabling efficient operations.
• Shape, Symmetries, and Structure: The Changing Role of Mathematics in Machine Learning Research — Mathematics' role in ML is shifting from theoretical guarantees to post-hoc explanations and high-level architectural guidance, leveraging advanced pure math.
• You Have Been Doing Geometry All Along — Data science workflows are fundamentally geometric, shaping how models perceive and learn structure.
• Every Supervised Machine Learning Model Is Linear — Feature engineering via geometric algebra can enable linear models to outperform complex ensembles like XGBoost.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Agus’s Substack.