John Carlin says, “‘Identifying variables that independently predict…’ is not a well-defined research task”

2026-05-02 · Source: Statistical Modeling, Causal Inference, and Social Science · Field: Science & Research — Mathematics & Computational Sciences, Research Methodology & Innovation · Depth: Advanced, short

Summary

John Carlin's recent work in epidemiological research methodology emphasizes classifying research questions into descriptive, predictive, and causal types to achieve clarity of purpose. The author of this analysis, while agreeing with the classification, suggests that descriptive and causal questions can be viewed as special cases of predictive questions, aligning with a broader data science perspective. Carlin also critiques the practice of dichotomizing research questions, such as "is this an independent prognostic factor?", advocating instead for assessing predictive value by comparing models with and without specific variables. The author supports Carlin's stance against dichotomization, noting their shared contributions to this theme in "Bayesian Data Analysis." However, the author expresses confusion regarding Carlin's specific method for assessing predictive value and questions Carlin's focus on "adjustment" debates, arguing that sharply defined research questions are universally beneficial, not just for statistical adjustments.

Key takeaway

For Data Scientists and Research Scientists designing epidemiological studies, prioritize framing research questions as descriptive, predictive, or causal to enhance clarity. Avoid dichotomous questions like "is X a prognostic factor?" and instead assess variable importance by comparing model performance with and without the variable. This approach aligns with robust statistical practices and helps avoid misinterpretations of "independent effects."

Key insights

Classifying research questions into descriptive, predictive, and causal types enhances clarity in epidemiological studies.

Principles

• Avoid dichotomous research questions.
• Assess predictive value by model comparison.

Method

Evaluate a variable's predictive value by comparing a model's performance with and without that variable, rather than relying solely on its "independent effect" in a multivariable regression.

In practice

• Frame questions as predictive, not dichotomous.
• Compare model performance for variable assessment.

Topics

• Research Question Classification
• Predictive Modeling
• Causal Inference
• Bayesian Data Analysis
• Multivariable Regression

Best for: Data Scientist, Research Scientist, AI Scientist

Related on AIssential

• “Statistics is widely understood to provide a body of techniques for ‘modeling data.'” — Clearly classifying research questions as descriptive, predictive, or causal is fundamental for robust statistical analysis.
• # Why Most Data Science Doesn’t Answer the Question You’re Asking — Distinguishing correlation from causation is fundamental for making effective, intervention-based decisions in data science.
• Conditional Hypothesis Generation for LLM-Based Text Analysis with Researcher-Specified Covariates — Conditional hypothesis generation uses covariates to find subgroup-specific language differences, avoiding confounds in LLM-based text analysis.
• A causal fused lasso for interpretable heterogeneous treatment effects estimation — The causal fused lasso adaptively estimates interpretable, piecewise constant heterogeneous treatment effects.
• The stupidity, the criminal vandalism, the wanton destruction of information involved in dichotomisation — Dichotomizing continuous outcomes in clinical trials leads to significant information loss and inflated sample sizes.
• Distributional Instrumental Variable Method — DIV uses generative modeling to estimate full causal effect distributions, overcoming traditional IV limitations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Statistical Modeling, Causal Inference, and Social Science.