Inverse Probability Weighting: Reweighting Instead of Discarding

2026-05-31 · Source: Data Science on Medium · Field: Technology & Digital — Data Science & Analytics, Artificial Intelligence & Machine Learning · Depth: Advanced, medium

Summary

Inverse Probability Weighting (IPW), the focus of Part 5 in an Applied Causal Inference series, is analyzed using the LaLonde job training dataset, which has a known true effect of \$1,794. Unlike Propensity Score Matching (PSM) which discarded 102 of 185 treated workers, IPW keeps all treated units and reweights control units to resemble the treated group. On the CPS-3 control group, IPW yielded an Average Treatment Effect on the Treated (ATT) of \$1,212, with a 95% CI of [−\$141, \$2,718]. This estimate is substantially closer to the benchmark and has a narrower interval than PSM's \$123–\$468. However, IPW on the full CPS group (15,992 workers) produced a wrong-signed ATT of −\$1,021, demonstrating failure when overlap is poor. The article also notes that AIPW, a doubly-robust estimator, performed worse than raw IPW on CPS-3, yielding \$476, due to misspecified outcome regression models. Effective Sample Size (ESS) is highlighted as the critical diagnostic for IPW.

Key takeaway

For data scientists or causal inference practitioners evaluating treatment effects, Inverse Probability Weighting (IPW) offers a powerful alternative to Propensity Score Matching. You should prioritize assessing covariate overlap using Effective Sample Size (ESS) before interpreting IPW results, as poor overlap leads to meaningless estimates. If using doubly-robust methods like AIPW, ensure your outcome regression model is well-specified for your data distribution, especially with skewed outcomes, to avoid adding noise.

Key insights

Inverse Probability Weighting improves causal estimates by reweighting controls, but requires good covariate overlap and correctly specified models.

Principles

• IPW's power comes from increased effective sample size.
• Overlap is the binding constraint for both IPW and PSM.
• Doubly-robust methods need at least one correct model.

Method

IPW uses logistic regression to compute propensity scores, then reweights control units (e(X)/(1−e(X)) for ATT) to match the treated group, keeping treated units at weight 1.

In practice

• Use Effective Sample Size (ESS) as IPW's primary diagnostic.
• Invest in flexible nonlinear outcome models for AIPW.
• Avoid IPW when propensity model AUC is near 1.0.

Topics

• Inverse Probability Weighting
• Causal Inference
• Propensity Score Matching
• Doubly Robust Estimation
• LaLonde Dataset
• Effective Sample Size

Code references

• Ajay-Deshpande/applied-causal-inference

Best for: Data Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Guide to Propensity Score Matching for Causal Inference to Estimate True Impact — PSM enables causal inference from observational data by statistically balancing treatment and control groups.
• Robustness of Refugee-Matching Gains to Off-Policy Evaluation Choices — Algorithmic refugee matching gains are robust across diverse off-policy evaluation methods, confirming original findings.
• Beyond Demand Estimation: Consumer Surplus Evaluation via Cumulative Propensity Weights — A new framework evaluates consumer surplus from AI pricing without explicit demand function estimation.
• Three-in-One World Model: Energy-Based Consistency, Prediction, and Counterfactual Inference for Marketing Intervention — A DBM-based world model with task-specific adapters unifies prediction, consistency, and counterfactual inference for marketing interventions.
• Model-Agnostic Meta Learning for Class Imbalance Adaptation — HAMR dynamically addresses class imbalance and data difficulty in NLP via adaptive weighting and neighborhood-aware resampling.
• Improving RCT-Based Treatment Effect Estimation Under Covariate Mismatch via Calibrated Alignment — CALM improves treatment effect estimation by aligning RCT and observational data in a shared embedding space.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Data Science on Medium.