Overfitting vs Underfitting: A Simple Explanation

2026-04-23 · Source: Deep Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Novice, quick

Summary

Machine learning models face a fundamental challenge in balancing overfitting and underfitting, which dictate a model's ability to generalize to unseen data. Underfitting occurs when a model is too simple, failing to capture underlying data patterns, resulting in poor performance on both training and test data due to high bias. Conversely, overfitting arises when a model is excessively complex, memorizing training data noise rather than generalizable trends, leading to excellent training performance but poor real-world accuracy due to high variance. Detecting these issues involves comparing training and validation accuracy; high errors in both suggest underfitting, while low training error and high validation error indicate overfitting. The goal is to achieve low and close errors for both, signifying a well-fitted model.

Key takeaway

For Data Scientists and Machine Learning Engineers building predictive models, understanding the bias-variance tradeoff is critical. If your model performs poorly on both training and test data, increase its complexity or add features. If it excels on training but fails on new data, simplify it, apply regularization, or gather more diverse data to improve generalization and avoid common pitfalls.

Key insights

Balancing model complexity is crucial to avoid underfitting (too simple) and overfitting (too complex) for effective generalization.

Principles

• High bias indicates underfitting.
• High variance indicates overfitting.
• The bias-variance tradeoff is fundamental.

Method

Compare training and validation errors: high errors in both signal underfitting; low training error with high validation error indicates overfitting; low and close errors mean a well-fitted model.

In practice

• Increase model depth to fix underfitting.
• Apply L1/L2 regularization to fix overfitting.
• Use cross-validation to prevent both.

Topics

• Overfitting
• Underfitting
• Machine Learning Models
• Bias-Variance Tradeoff
• Model Generalization

Best for: AI Student, Data Scientist, Machine Learning Engineer

Related on AIssential

• The first thing you need to understand when you start with AI — Effective machine learning models balance bias (underfitting) and variance (overfitting) to generalize well.
• The Bias–Variance Tradeoff: The Real Reason Your Model Fails — Understanding the bias-variance tradeoff is crucial for diagnosing and resolving common machine learning model failures.
• Regularisation in Neural Networks — Regularization techniques are crucial for neural networks to prevent overfitting and improve generalization on unseen data.
• Bagging vs Boosting: Variance vs Bias — Bagging and Boosting are opposite ensemble methods that combine weak learners into strong models, with Bagging reducing variance and Boosting reducing bias.
• Machine Learning | Ridge Regression-L2 — Ridge Regression uses L2 regularization to stabilize coefficients, reduce overfitting, and manage multicollinearity in linear models.
• A Regularization-Sharpness Tradeoff for Linear Interpolators — A regularization-sharpness tradeoff replaces bias-variance in overparameterized linear models, decomposing model selection into alignment and local perturbation effects.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Deep Learning on Medium.