Linear Algebra for Machine Learning — Part 2: The Dot Product and the Idea of Similarity

2026-07-19 · Source: Data Science on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mathematics & Computational Sciences · Depth: Novice, long

Summary

Part 2 of a linear algebra series introduces the dot product and cosine similarity, fundamental tools for measuring agreement between vectors in machine learning. The dot product quantifies how much two vectors point in the same direction, yielding a big positive number for alignment, zero for perpendicularity (unrelated), and a negative number for opposition. Geometrically, this is visualized as one vector's shadow on another. While the dot product considers both length and direction, cosine similarity normalizes vectors to unit length, focusing exclusively on direction. This normalization prevents longer vectors from artificially inflating similarity scores. These concepts are critical for applications such as search engines, face unlock, recommendation systems, and the "attention" mechanism in large language models like ChatGPT, enabling computers to compare and relate diverse data points represented as vectors.

Key takeaway

For Machine Learning Engineers designing similarity features or debugging model behavior, understanding the distinction between dot product and cosine similarity is crucial. You should prioritize cosine similarity when vector length is irrelevant to meaning, such as in text search or recommendation systems, to avoid length-based biases. This ensures your models accurately capture directional agreement, leading to more robust and interpretable similarity metrics in applications like face recognition or LLM attention mechanisms.

Key insights

The dot product and cosine similarity quantify vector agreement, forming the basis for AI's understanding of similarity.

Principles

• Dot product quantifies vector agreement.
• Cosine similarity isolates directional agreement.
• Perpendicular vectors signify no relationship.

Method

Calculate dot product by multiplying corresponding vector components and summing. Compute cosine similarity by dividing the dot product by the product of the vectors' lengths.

In practice

• Implement cosine similarity for search.
• Apply vector comparison for face unlock.
• Utilize dot products in neural network layers.

Topics

• Linear Algebra
• Dot Product
• Cosine Similarity
• Vector Embeddings
• Machine Learning Foundations
• AI Applications

Best for: AI Student, Machine Learning Engineer, Data Scientist

Related on AIssential

• Cosine Similarity is Just Direction, Not Distance — Cosine similarity accurately gauges semantic similarity by focusing on vector direction, not magnitude, essential for high-dimensional embeddings.
• Linear Algebra For AI (Part 1: The Building Blocks) — Linear algebra's core concepts, like vectors as arrows and matrices as transformations, underpin AI mechanisms such as embeddings and attention.
• Linear Regression Is Actually a Projection Problem, Part 1: The Geometric Intuition — Understanding vectors, dot products, and projections provides a geometric intuition for linear regression.
• Linear Algebra for Machine Learning — Part 1: Why Vectors Exist — Vectors are fundamental AI building blocks, representing both magnitude and direction for diverse data types.
• Interpretable Discriminative Text Representations via Agreement and Label Disentanglement — Interpretable text features require both conceptual clarity via inter-annotator agreement and label disentanglement.
• When LLMs Answer the Wrong Question — Drift, using geometric algebra, measures relational faithfulness in AI answers, distinguishing it from mere topical relevance.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Data Science on Medium.