Correlation vs. Causation: Measuring True Impact with Propensity Score Matching

2026-04-22 · Source: Towards Data Science · Field: Technology & Digital — Data Science & Analytics, Artificial Intelligence & Machine Learning · Depth: Intermediate, medium

Summary

Propensity Score Matching (PSM) is a statistical technique used to determine if a specific "treatment" causes a particular result, especially when a perfectly randomized experiment is not feasible. It addresses the problem of comparing groups with pre-existing differences by finding "statistical twins" in the data—individuals who are similar in observed characteristics but differ only in whether they received the treatment. The article demonstrates a Python implementation of PSM using a generated dataset of 1000 rows with variables like age, past expenses, and mobile device use, aiming to assess the impact of an advertising campaign. The process involves calculating propensity scores via Logistic Regression, matching pairs using NearestNeighbors with a caliper threshold, and evaluating the match quality using Standardized Mean Difference (SMD). The example concludes by calculating the difference in means, performing a T-test, and computing Cohen's d to assess the treatment's effect size.

Key takeaway

For Data Scientists analyzing the causal impact of interventions without randomized control, Propensity Score Matching offers a robust method to create comparable groups. You should apply this technique to observational data to isolate treatment effects from confounding variables. This approach allows you to confidently attribute changes in outcomes, such as customer spending, directly to the intervention, even when a traditional A/B test was not conducted.

Key insights

Propensity Score Matching isolates treatment effects by creating statistically balanced groups from observational data.

Principles

• Balance covariates to attribute outcome differences to treatment.
• Small Standardized Mean Difference (SMD) indicates good balance.

Method

Implement PSM by calculating propensity scores with Logistic Regression, matching pairs using NearestNeighbors, applying a caliper for quality, and evaluating balance with SMD before assessing treatment effects.

In practice

• Use `sklearn.linear_model.LogisticRegression` for propensity scores.
• Employ `sklearn.neighbors.NearestNeighbors` for matching.
• Calculate Cohen's d for effect size interpretation.

Topics

• Propensity Score Matching
• Causal Inference
• Logistic Regression
• Nearest Neighbors Algorithm
• Standardized Mean Difference

Code references

• gurezende/Propensity-Score-Matching

Best for: Data Scientist, Consultant, Analytics Engineer

Related on AIssential

• Guide to Propensity Score Matching for Causal Inference to Estimate True Impact — PSM enables causal inference from observational data by statistically balancing treatment and control groups.
• Topological Ignorability for Structural Causal Effects Beyond Means — Topological-geometrical causal metrics reveal structural outcome changes missed by mean-based average treatment effects.
• A causal fused lasso for interpretable heterogeneous treatment effects estimation — The causal fused lasso adaptively estimates interpretable, piecewise constant heterogeneous treatment effects.
• # Why Most Data Science Doesn’t Answer the Question You’re Asking — Distinguishing correlation from causation is fundamental for making effective, intervention-based decisions in data science.
• Balanced Twins: Causal Inference on Time Series with Hidden Confounding — B-Twin robustly estimates treatment effects in time series by matching latent representations and propensity scores under hidden confounding.
• Proximal Mediation Analysis with Hidden Recanting Witnesses — Proximal causal inference identifies path-specific effects despite hidden recanting witnesses, enabling robust mediation analysis.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards Data Science.