You’ve Been Using Z-Scores. But Do You Actually Know What They’re Saying?

2026-04-25 · Source: Towards AI - Medium · Field: Technology & Digital — Data Science & Analytics, Artificial Intelligence & Machine Learning · Depth: Novice, medium

Summary

A Z-score quantifies how far a data point deviates from its mean, measured in standard deviations, providing context beyond raw values. For instance, a student scoring 78 in biology with a class average of 85 and standard deviation of 6 yields a Z-score of -1.17, while a student scoring 62 in physics with an average of 50 and standard deviation of 8 yields a Z-score of +1.50. This method reveals the physics student performed better relative to their class. Z-scores are crucial in feature scaling for machine learning, outlier detection (often flagging values beyond ±3 standard deviations), and comparing data across different distributions. However, their reliability diminishes with heavily skewed data, necessitating visualization of data distribution before application.

Key takeaway

For data scientists and analysts evaluating disparate datasets or preparing features for machine learning, understanding Z-scores is critical. You should visualize your data's distribution before relying on Z-scores for anomaly detection, especially with skewed data, to avoid misinterpreting normal features as outliers. Remember that a high Z-score indicates statistical unusualness, but your domain knowledge is essential to determine its practical significance.

Key insights

Z-scores contextualize data points by measuring their distance from the mean in standard deviation units.

Principles

• Measure deviation in standard deviations, not raw points.
• Statistical unusualness is not practical significance.

Method

Calculate Z-score as (X - μ) / σ to normalize data, where X is the value, μ is the mean, and σ is the standard deviation. Use thresholds like ±2.5 or ±3 for flagging outliers.

In practice

• Use Z-scores for feature scaling in ML models.
• Apply Z-scores for anomaly and fraud detection.
• Compare values from different datasets using Z-scores.

Topics

• Z-score
• Standard Deviation
• Outlier Detection
• Feature Scaling
• Data Standardization

Best for: Data Scientist, Machine Learning Engineer, Data Analyst

Related on AIssential

• We Used 5 Outlier Detection Methods on a Real Dataset: They Disagreed on 96% of Flagged Samples — Different outlier detection methods identify distinct types of "unusual" data points, leading to low agreement.
• Normalization vs Standardization - Explained — Feature scaling prevents algorithms from misinterpreting large numerical ranges as greater importance, improving model training efficiency.
• Hypothesis Testing Is Just a Distance — Hypothesis testing fundamentally measures the distance of an observed result from a chance expectation, expressed in standard deviations.
• A Visual Explanation of Linear Regression — Visual, practical, and reproducible methods are key to understanding and applying linear regression effectively.
• The Data Points That Don’t Belong -And Why They Matter More Than You Think — Modern outlier detection leverages diverse techniques to find anomalies in complex, high-dimensional, and streaming datasets.
• Building Robust Credit Scoring Models (Part 3) — Proper data splitting and preprocessing are crucial for building generalizable and stable credit scoring models.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards AI - Medium.