Boosting with List-Decodable Codes

2026-07-08 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, medium

Summary

A new boosting algorithm, "Boosting with List-Decodable Codes," significantly reduces the round complexity of learning algorithms for specific concept classes. Traditional boosting methods, like AdaBoost, require O(log(1/epsilon)/gamma^2) calls to a gamma-advantage weak learner, a bound previously considered optimal. This new approach circumvents that lower bound for concept classes closed under O(log(1/gamma))-XOR. It achieves strong learning with only O(log(1/epsilon)) calls to the weak learner and ~O(log(1/epsilon)/gamma^2) additional labeled samples. The method leverages a novel connection to list-decodable codes, treating the target function as a message and the weak hypothesis as a corrupted codeword. A variant also enables uniform-distribution boosting with just one weak learner call and ~O(log(1/gamma)/epsilon) additional samples for classes closed under O(log(1/gamma)/epsilon)-XOR, offering substantial improvements over prior distribution-specific techniques.

Key takeaway

For AI scientists designing boosting algorithms, this research indicates that the long-held O(log(1/epsilon)/gamma^2) round complexity is not a universal limit. If your concept class exhibits mild closure properties, such as O(log(1/gamma))-XOR, you can achieve significantly fewer weak learner calls. Consider applying list-decodable code principles to reduce computational rounds, especially for distribution-specific learning, where a single weak learner call might suffice. This could optimize resource usage for structured data problems.

Key insights

Existing boosting round complexity lower bounds can be circumvented for structured concept classes.

Principles

• Boosting round complexity is not universally optimal.
• Closure properties enable more efficient boosting.
• List-decodable codes can enhance boosting.

Method

The algorithm encodes the target function, runs a weak learner on the encoding, uses a list decoder to generate candidate hypotheses, then identifies the strong hypothesis with additional samples.

In practice

• Apply to concept classes closed under O(log n)-XOR.
• Consider for uniform-distribution specific learning tasks.
• Evaluate for classes like small, constant-depth circuits.

Topics

• Boosting Algorithms
• List-Decodable Codes
• Concept Classes
• Round Complexity
• Weak Learning
• XOR Closure

Best for: Research Scientist, AI Scientist

Related on AIssential

• An Information-Theoretic Criterion for Efficient Data Synthesis — Synthetic data is effective only when external signals keep the training loop "information-open."
• Bagging vs Boosting: The Complete Beginner’s Guide to Ensemble Learning in Machine Learning — Ensemble Learning combines multiple models to improve prediction, with Bagging and Boosting as core techniques.
• A unified complexity bound for logconcave sampling — A new unified complexity bound for logconcave sampling achieves nearly tight convergence using In-and-Out and exponential lifting.
• Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation — A new LTR framework offers metric-agnostic optimization for improved ranking generalization and stability.
• Quantization robustness from dense representations of sparse functions in high-capacity kernel associative memory — KLR-trained Hopfield networks achieve quantization robustness via a "sparse function, dense representation" principle.
• Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity — O-minimal definability ensures finite PAC sample complexity for a wide range of feedforward neural networks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.