When “Garbage In, Garbage Out” Gets It Wrong

2026-05-21 · Source: The Data Exchange · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Research Methodology & Innovation · Depth: Advanced, extended

Summary

Terrence Lee-St. John, founder of Enli and lead author of "From Garbage to Gold: A Data Architectural Theory of Predictive Robustness," challenges the "garbage in, garbage out" mantra for predictive models. His work explains why models trained on traditionally "garbage" tabular data can achieve state-of-the-art performance. The core insight is that observed data are shadows of underlying latent drivers, and wide, messy datasets with redundant signals can effectively triangulate and recover these latent states. This theory is particularly significant for regulated sectors like healthcare, where empirical evidence of model effectiveness on dirty data often faces pushback. Traditional data cleaning, while intuitive, is expensive, time-consuming, and reduces data dimensionality, potentially hindering the recovery of latent signals. Lee-St. John's research suggests that expanding predictor sets, even with errors, can improve prediction by overcoming structural uncertainty. Enli aims to operationalize this data-centric approach, building warehouse-native, stable, and explainable predictive systems that leverage high-dimensional, error-prone data. For instance, a project at Cleveland Clinic Abu Dhabi reduced 32,000 predictors to 900-4,000 columns while maintaining predictive power.

Key takeaway

For data scientists building predictive models, especially in regulated sectors like healthcare, reconsider the default assumption that extensive data cleaning is always optimal. Your focus should shift from pristine individual features to maximizing data breadth and ensuring comprehensive coverage of underlying latent drivers. Over-cleaning can inadvertently reduce the dimensionality and redundancy crucial for robust predictions. Instead, explore expanding your predictor sets, even with error-prone data, and prioritize features based on their information gain to model residuals. This approach can yield more stable and explainable systems.

Key insights

Wide, messy tabular datasets can produce robust predictions by using redundant signals to recover underlying latent drivers, defying "garbage in, garbage out."

Principles

• Observed data are shadows of underlying latent drivers.
• Redundancy in data aids efficient latent state recovery.
• Data quality is about comprehensive latent driver coverage.

Method

Correlate potential variables with model residuals to identify those providing the most information gain, iteratively expanding the predictor space.

In practice

• Expand predictor sets, even with errors, for better latent signal recovery.
• Prioritize features by correlating with model residuals for information gain.
• Explain models at the latent layer, not raw feature layer.

Topics

• Predictive Robustness
• Tabular Data
• Latent Variables
• Data Quality
• Feature Selection
• Data-centric AI
• Healthcare AI

Best for: Research Scientist, Machine Learning Engineer, AI Scientist, Data Scientist, Director of AI/ML

Related on AIssential

• Why Data Cleaning Takes Longer Than the Model In AI — High-quality data preparation is more critical for machine learning success than complex algorithms.
• Cellwise Outliers — Cellwise outliers, individual anomalous data points, require distinct robust statistical methods, especially in high-dimensional data.
• Double descent for least-squares interpolation on contaminated data: A simulation study — The least-squares interpolator exhibits double descent and can outperform robust methods on contaminated data with high overparameterization.
• A Visual Explanation of Linear Regression — Visual, practical, and reproducible methods are key to understanding and applying linear regression effectively.
• I Improved My Model… But My Score Got Worse — Cross-validation improvements don't guarantee generalization; simpler, domain-driven features often outperform complex ones.
• Neural Bayesian Anomaly Mitigation: A Robust Loss that Doubles as an Unsupervised Contamination Classifier — NBAM is a robust loss that simultaneously classifies contamination using a Bayesian latent-switch mixture model.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Data Exchange.