Corruptions of Supervised Learning Problems: Typology and Mitigations

2025-12-31 · Source: JMLR · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, quick

Summary

A new general theory of corruption is presented by Laura Iacovissi, Nan Lu, and Robert C. Williamson in their 2026 work, addressing widespread data collection issues in supervised learning. This theory, detailed across 27(72):1−73, uses Markov kernels to unify existing corruption models and establish a consistent nomenclature. It provides a provably exhaustive framework for distinguishing corruption types and systematically analyzes their consequences on learning tasks by considering Bayes risks. The research highlights that label corruptions primarily affect the loss function, while attribute corruptions additionally influence the hypothesis class. Furthermore, it expands existing loss-correction methods for label corruption to handle dependent types, proposing a new paradigm with weaker requirements and offering loss correction formulas for attribute and joint corruption cases.

Key takeaway

For machine learning engineers dealing with noisy datasets, this research provides a unified framework to identify and mitigate various corruption types more effectively. It clarifies that attribute corruptions are fundamentally different from label corruptions, impacting the hypothesis class, not just the loss function. You should consider adopting the proposed generalized loss-correction methods and the new corruption-corrected learning paradigm to build more robust and accurate models, especially when facing complex, dependent data corruptions.

Key insights

A general theory unifies data corruption types and provides systematic mitigation strategies for supervised learning.

Principles

• Corruption can be modeled via Markov kernels.
• Label corruptions affect loss; attribute corruptions affect hypothesis class.
• Existing loss-correction needs generalization for broader corruption types.

Method

The paper develops a general theory of corruption using Markov kernels to construct an exhaustive framework, analyze Bayes risks, and derive generalized loss-correction formulas for attribute and joint corruptions.

In practice

• Apply generalized loss-correction for attribute corruption.
• Distinguish corruption types using the new framework.
• Evaluate Bayes risks in corrupted scenarios.

Topics

• Supervised Learning
• Data Corruption
• Markov Kernels
• Loss Correction
• Bayes Risk
• Machine Learning Theory

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Neural Bayesian Anomaly Mitigation: A Robust Loss that Doubles as an Unsupervised Contamination Classifier — NBAM is a robust loss that simultaneously classifies contamination using a Bayesian latent-switch mixture model.
• Noise-Aware Framework for Correcting Corrupted Labels — CANOLA corrects corrupted labels by estimating noise and iteratively refining them for robust model training.
• Distributionally Robust Set Representation Learning Under Inference-Time Element Corruption — SW-DRSO enhances set representation learning robustness by optimizing for worst-case inference-time corruption using a barycentric adversary.
• Effects of Training Data Quality on Classifier Performance — Classifier performance degrades universally with training data quality, leading to coincidentally correct but flawed decisions.
• When “Garbage In, Garbage Out” Gets It Wrong — Wide, messy tabular datasets can produce robust predictions by using redundant signals to recover underlying latent drivers, defying "garbage in, garbage out."
• Practical estimation of the optimal classification error with soft labels and calibration — Isotonic calibration enables consistent Bayes error estimation even with corrupted soft labels, provided their order is preserved.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by JMLR.