Common Data Science Mistakes and How to Avoid Them

2026-07-26 · Source: Data Science on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, medium

Summary

This article outlines nine common data science mistakes that often produce confident, plausible-looking, yet incorrect, results. It details errors such as data leakage, where information from outside training data influences a model, and testing on training data, which overstates real performance. Other critical errors include ignoring class imbalance, mistaking correlation for causation, and undetected overfitting. The analysis also covers inappropriate handling of missing data, misleading visualizations through truncated axes, and drawing conclusions from too small a sample. Finally, it addresses the failure to check underlying assumptions of chosen techniques, like using linear regression for non-linear relationships. The article stresses that these issues are dangerous because they do not generate errors, necessitating deliberate, active checking.

Key takeaway

For data scientists and machine learning engineers preparing to deploy models or submit analyses, you must proactively scrutinize your work for common pitfalls. Implement a rigorous pre-submission checklist to verify data integrity, model evaluation, and visualization honesty. This prevents quietly flawed models or misleading reports from reaching production, safeguarding your research's trustworthiness and avoiding real-world consequences from undetected errors.

Key insights

Common data science mistakes often produce plausible but incorrect results, requiring deliberate checks.

Principles

• Trustworthy data science requires active error checking.
• Plausible output does not guarantee correctness.
• Understand technique assumptions before application.

Method

The article proposes a nine-point pre-submission checklist to identify and prevent common data science errors, including verifying data availability at prediction time, checking class balance, and comparing training/test performance.

In practice

• Implement a train-test split or cross-validation.
• Compare training and test scores side-by-side.
• Visually inspect data for technique assumption validity.

Topics

• Data Leakage
• Model Evaluation
• Class Imbalance
• Causal Inference
• Data Visualization Best Practices
• Missing Data Imputation

Best for: Data Scientist, Machine Learning Engineer, AI Student

Related on AIssential

• Python for Data Science — Why Data Leakage Destroys Models — Data leakage inflates model performance during development, leading to catastrophic failure in real-world deployment due to access to future information.
• Group-by statements that save the day — Simple data exploration, like GROUP BY, often reveals critical issues that complex ML models miss, emphasizing human judgment.
• Is this data actually good, or does it just look good? — Data quality audits are crucial for machine learning, revealing hidden issues like misspellings, ambiguity, and train/test leakage that impact model performance.
• Navigating Common Pitfalls in Data Science: Lessons from Pierpaolo Hipolito - ML 183 — Understanding data paradoxes and causal reasoning is crucial for building accurate and interpretable machine learning models.
• Do a sanity check on your experiments — Manual sanity checks on data, model outputs, and evaluations are crucial for detecting common experimental bugs.
• Handling Imbalanced Data — Imbalanced data causes misleading model accuracy, necessitating specific metrics and techniques for reliable performance.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Data Science on Medium.