Geometry Without Coordinates

2026-01-11 · Source: Agus’s Substack · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mathematics & Computational Sciences · Depth: Advanced, long

Summary

This article introduces kernels, affinities, and graphs as geometric tools for machine learning when objects' structure is relational rather than coordinate-based. It distinguishes between distance and similarity functions, emphasizing that similarity, often task-relevant, asks what objects have in common. Kernels are presented as generalized inner products, computing alignment in a hidden, possibly high-dimensional, feature space without explicit transformation, exemplified by linear, polynomial, and Gaussian (RBF) kernels. The article explains how kernels enable working with complex geometries, such as separating non-linearly separable data, and extends this concept to learned kernels like self-attention in Transformers, which are dynamic and asymmetric. It also discusses bandwidth as a critical geometric choice, affinity matrices, neighborhood graphs, and spectral methods like Laplacian eigenvectors and diffusion maps for uncovering global structure from local relations.

Key takeaway

For machine learning engineers developing models where raw coordinate proximity is insufficient, consider adopting kernel-based or graph-based geometric approaches. Your choice of kernel, similarity function, or graph construction (e.g., k-NN, ε-neighborhood) is a fundamental modeling commitment that defines how your system perceives object relationships. Experiment with different kernels and bandwidths, treating them as geometric decisions, to uncover latent structures that improve model performance and interpretability.

Key insights

Relational geometry, defined by kernels and graphs, often reveals structure that raw coordinate-based methods obscure.

Principles

• Similarity is distinct from distance.
• Kernels define geometry in implicit feature spaces.
• Bandwidth is a geometric choice, not a mere parameter.

Method

Kernels compute inner products in a feature space via k(x, z) = ⟨φ(x), φ(z)⟩, enabling implicit transformation. Spectral methods use graph Laplacians to embed local relations into global structures.

In practice

• Use cosine similarity for directional alignment.
• Employ Gaussian kernels for smooth, localized similarity.
• Apply Nystrom approximation for scalable kernel methods.

Topics

• Coordinate-Free Geometry
• Kernel Functions
• Affinity Graphs
• Spectral Analysis
• Self-Attention Mechanism

Code references

• asudjianto-xml/geomlearn

Best for: AI Scientist, Machine Learning Engineer, Data Scientist

Related on AIssential

• The Geometry Hidden in Every Model — Learning in supervised models is the discovery of a geometric kernel that defines similarity and weights training labels.
• Heat and Matérn Kernels on Matchings — A framework extends heat and Matérn kernels to matchings, using zonal polynomials for efficient evaluation.
• The Kernel as the Normal Form — Most machine learning methods are variations of a kernel machine, with geometry (the kernel) as the primary choice.
• Top 20 K-Nearest Neighbors (KNN) Interview Questions and Answer (Part 1 of 2) — KNN is a lazy, non-parametric algorithm predicting outcomes based on the K most similar data points.
• A Practitioner's Guide to Kolmogorov-Arnold Networks — KANs offer enhanced expressivity and interpretability over MLPs by using learnable univariate basis functions on network edges.
• Neural Networks on Symmetric Spaces of Noncompact Type — A unified point-to-hyperplane distance enables novel neural network architectures on noncompact symmetric spaces.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Agus’s Substack.