LLM Themes Are Not Observations

2026-05-21 · Source: Towards Data Science · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, long

Summary

LLM-extracted themes, often used in causal analyses, are not direct observations of customer attributes but "generated variables" derived from a self-selected subset of customer behavior. This introduces significant bias, particularly when preprocessing steps like filling NULL values for customers without text. Four critical issues arise: selection bias, where themes exist only for customers who generated text; timing bias, misclassifying pre-treatment, concurrent, or post-treatment text; measurement error, as LLM labels are noisy proxies whose accuracy can differ across treatment arms; and role misclassification within the causal graph. A synthetic example demonstrates how a naive regression with a "bill_shock" theme can flip the sign of a retention offer's effect on churn from a true -0.50 to a misleading +0.12.

Key takeaway

For Data Scientists or ML Engineers building causal models with LLM-derived features, recognize these are generated variables, not direct observations. Your analysis must explicitly account for selection, timing, and measurement error to avoid biased conclusions, such as incorrectly inferring treatment effects. Always stress-test your models by removing text-derived variables to assess result stability and ensure your causal claims are defensible.

Key insights

LLM-extracted themes are generated variables, not direct observations, introducing bias in downstream causal models.

Principles

• Generated variables inherit complex data-generating processes.
• Causal role of text-derived variables depends on generation timing.
• Selection into text generation redefines the analysis population.

In practice

• Trace the structural footprint of generated variables.
• Explicitly state assumptions for customers without text.
• Stress-test models by removing text-derived variables.

Topics

• Causal Inference
• LLM Applications
• Generated Variables
• Selection Bias
• Measurement Error
• Data Preprocessing

Best for: AI Engineer, NLP Engineer, AI Architect, Data Scientist, Machine Learning Engineer, Director of AI/ML

Related on AIssential

• Conditional Hypothesis Generation for LLM-Based Text Analysis with Researcher-Specified Covariates — Conditional hypothesis generation uses covariates to find subgroup-specific language differences, avoiding confounds in LLM-based text analysis.
• # Why Most Data Science Doesn’t Answer the Question You’re Asking — Distinguishing correlation from causation is fundamental for making effective, intervention-based decisions in data science.
• Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact — Apparent psychological profiles of LLMs are primarily measurement artifacts driven by directional response bias, not inherent traits.
• Claude Can’t Run Regression. A 200-Year-Old Theorem Proves It Never Will. — LLMs cannot perform true statistical inference, lacking the human judgment and domain knowledge essential for interpreting assumptions.
• Test Time Training for Supervised Causal Learning — TTT-SCL improves Supervised Causal Learning's real-world applicability by dynamically generating test-aligned training sets.
• CauTion: Knowing When to Trust LLMs for Ensemble Causal Discovery — CauTion integrates LLM domain knowledge with statistical ensembles using trust calibration to improve causal discovery accuracy.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards Data Science.