Vector Norms - Explained

2026-03-22 · Source: DataMListic · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Novice, quick

Summary

Vector norms provide various methods for measuring the "size" or magnitude of a vector, extending beyond the single intuitive answer. The L2 norm, or Euclidean norm, calculates magnitude using the Pythagorean theorem, summing the squares of components and taking the square root. The L1 norm, also known as the Manhattan norm, sums the absolute values of components, representing distance on a grid. The L-infinity norm identifies the maximum absolute value among a vector's components. These three norms are specific instances of the general P-norm formula, which involves raising each component's absolute value to the power P, summing them, and then taking the P-th root. Geometrically, the set of all vectors with a norm of 1 varies by norm type, forming a circle for L2, a diamond for L1, and a square for L-infinity, demonstrating how P-norms smoothly transition between these shapes. In machine learning, norms are crucial for regularization, with L1 penalties promoting sparsity by driving some weights to zero, while L2 penalties keep all weights small but non-zero.

Key takeaway

For data scientists and machine learning engineers selecting regularization techniques, understanding vector norms is critical. Your choice between L1 (Manhattan) and L2 (Euclidean) regularization directly impacts model sparsity and weight distribution. Opt for L1 when feature selection is desired, as it pushes less important weights to zero. Choose L2 to keep all weights small and prevent large individual weights, generally leading to more robust models.

Key insights

Vector norms offer diverse ways to quantify vector magnitude, each with distinct mathematical and geometric properties.

Principles

• Vector "size" depends on the chosen norm.
• P-norms generalize L1, L2, and L-infinity norms.
• Norms influence regularization in ML models.

In practice

• Use L1 regularization for feature selection.
• Apply L2 regularization to prevent overfitting.
• Visualize norm unit balls to understand geometry.

Topics

• Vector Norms
• L1 Regularization
• L2 Regularization
• Sparsity
• Machine Learning Regularization

Best for: Machine Learning Engineer, Data Scientist, AI Student

Related on AIssential

• L2 Regularization Is Secretly Bayesian — L2 regularization is mathematically equivalent to applying a Gaussian prior in Bayesian maximum a posteriori estimation.
• Normalization vs Standardization - Explained — Feature scaling prevents algorithms from misinterpreting large numerical ranges as greater importance, improving model training efficiency.
• The Geometry Underneath the Algebra — A small set of geometric ideas forms the structural foundation of classical machine learning and statistics.
• Which Regularizer Should You Actually Use? Lessons from 134,400 Simulations — Regularizer choice for linear models depends on optimization objective and data characteristics like sample size and multicollinearity.
• From Generalist to Specialist Representation — Task-relevant latent representations are identifiable nonparametrically, enabling robust specialization from generalist models.
• S2MAM: Semi-supervised Meta Additive Model for Robust Estimation and Variable Selection — S²MAM uses bilevel optimization for robust semi-supervised learning, identifying variables and updating similarity for interpretable predictions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by DataMListic.