The Chi-Squared Test : Are Two Distributions the Same? (with Python Example)

2026-01-19 · Source: Steve Brunton · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, long

Summary

The Chi-Squared test is a statistical method used to determine if two distributions are statistically the same, often applied to assess the goodness-of-fit between observed data and a theoretical probability distribution. The process involves binning observed data into categories and comparing these observed counts (O_i) against expected counts (E_i) derived from the hypothesized distribution. A test statistic, X^2 = \sum (O_i - E_i)^2 / E_i, is calculated, which follows a Chi-Squared distribution with D degrees of freedom, where D = (number of bins) - (number of fit parameters) - 1. This test statistic is then used in hypothesis testing: a null hypothesis states the distributions are the same, and a rejection region is established based on a chosen significance level (e.g., p=0.05). If the calculated X^2 falls outside this region, the null hypothesis is rejected, indicating the distributions are different. The article demonstrates this with a Python example using alpha particle emission data, testing its consistency with a Poisson distribution.

Key takeaway

For Data Scientists or Machine Learning Engineers evaluating model fits, the Chi-Squared test provides a robust method to quantitatively assess if your observed data is consistent with a chosen probability distribution. You should calculate the test statistic and its p-value to determine if your fitted model adequately represents the underlying data, ensuring your assumptions about data distribution are statistically sound. Remember to adjust binning to ensure sufficient counts per bin for valid results.

Key insights

The Chi-Squared test assesses if observed data aligns with a theoretical distribution by comparing binned counts.

Principles

• The Chi-Squared test statistic follows a Chi-Squared distribution.
• Degrees of freedom D = bins - parameters - 1.
• Test works best when bin counts are > 5.

Method

Bin observed data and calculate expected counts from a fitted distribution. Compute the Chi-Squared test statistic. Compare this statistic to a Chi-Squared distribution with appropriate degrees of freedom to determine if the null hypothesis (distributions are the same) can be rejected.

In practice

• Use for goodness-of-fit testing.
• Apply to compare two data sets' distributions.
• Group small bins to meet count > 5 requirement.

Topics

• Chi-Squared Test
• Hypothesis Testing
• Poisson Distribution
• Goodness of Fit
• Statistical Methods

Best for: Data Scientist, Machine Learning Engineer, AI Student

Related on AIssential

• The Chi-Squared Test is Just Squared Normals — The chi-squared test assesses if observed categorical frequencies significantly differ from expected values.
• Hypothesis Testing Revisited: Normal, t, and Chi-Squared Distribution Tests — Hypothesis testing follows a consistent procedure, adapting test statistics and distributions based on data characteristics and the hypothesis type.
• Chi-Square Test for Beginners — Complete Guide with Formula, Types, Example & Python Code — The Chi-Square test assesses if observed categorical data patterns are statistically significant or random.
• Properties of Chi-Squared and Student's t Distributions — Chi-Squared and Student's t distributions are fundamental for hypothesis testing, especially with small samples or unknown variances.
• Mathematics for Data Science #5: Probability Distributions (The Real World Behaves This Way) — Understanding specific probability distributions is crucial for accurately modeling real-world data in data science.
• The Essence of Linear Regression — Linear regression fits a line to data, using least squares to minimize prediction errors and R-squared/P-value to quantify confidence.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Steve Brunton.