Linear Algebra for Machine Learning — Part 1: Why Vectors Exist

2026-07-19 · Source: Machine Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Novice, medium

Summary

The article introduces vectors as fundamental building blocks for AI, explaining their core concept as entities possessing both magnitude (how much) and direction (which way). It illustrates this with examples like wind and navigation, contrasting vectors with scalars which only have magnitude. The text details how vectors are represented numerically (e.g., (3,4)), how their length (magnitude) is calculated using the Pythagorean theorem (e.g., √(3² + 4²) = 5), and introduces unit vectors as representations that preserve direction while normalizing length to 1. The article emphasizes vectors' pervasive role in AI, where everything from photos to words is converted into high-dimensional vectors for machine learning, and discusses common misconceptions, historical context, and real-world applications like face unlock and recommendation systems.

Key takeaway

For machine learning engineers building AI systems, understanding vectors as combined magnitude and direction is crucial. This foundational knowledge clarifies why data like images or text must be vectorized, enabling you to correctly interpret embeddings and similarity metrics. Recognize that unit vectors isolate direction for comparison, while scalar multiplication can flip orientation. Grasping these basics prevents misinterpretations of model behavior and prepares you for advanced concepts like cosine similarity.

Key insights

Vectors are fundamental AI building blocks, representing both magnitude and direction for diverse data types.

Principles

• A vector always combines magnitude and direction.
• Length and direction are independent properties of a vector.
• Unit vectors normalize length to 1, preserving only direction.

Method

Vector length is calculated using the Pythagorean theorem (√(x² + y²)). A unit vector is created by dividing a vector by its length.

In practice

• Convert diverse data (images, words) into vectors for AI processing.
• Use unit vectors to compare directional similarity, ignoring magnitude.
• Recognize vector length can imply information like word frequency in some models.

Topics

• Linear Algebra
• Vectors
• Machine Learning Foundations
• AI Embeddings
• Unit Vectors
• Vector Magnitude

Best for: AI Student, Machine Learning Engineer, Data Scientist

Related on AIssential

• Linear Algebra For AI (Part 1: The Building Blocks) — Linear algebra's core concepts, like vectors as arrows and matrices as transformations, underpin AI mechanisms such as embeddings and attention.
• How Linear Algebra Powers Machine Learning (ML) — Linear algebra underpins machine learning by transforming raw data into computable numerical structures and enabling efficient operations.
• Linear Algebra for Machine Learning — Part 2: The Dot Product and the Idea of Similarity — The dot product and cosine similarity quantify vector agreement, forming the basis for AI's understanding of similarity.
• Vector Embeddings — Vector embeddings transform complex data into numerical vectors, enabling machine learning to process semantic relationships.
• Notebook: Introduction to Linear Algebra #1 — Linear equations graph as lines; systems find solutions at intersections, scaling to complex data via linear algebra.
• You Can Fit a Million Nearly-Perpendicular Arrows in 768 Dimensions. — High-dimensional spaces enable millions of nearly-perpendicular vectors, explaining why embeddings effectively represent numerous distinct ideas.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning on Medium.