Does YOLO Really Need to See Every Training Image in Every Epoch?

2026-03-18 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Advanced, medium

Summary

YOLO detectors, despite their fast inference, suffer from time-consuming training due to processing every image in every epoch, even when many are already learned. Researchers propose an Anti-Forgetting Sampling Strategy (AFSS) to address this by dynamically selecting training images. AFSS measures "learning sufficiency" for each image using detection recall and precision, categorizing them as easy, medium, or hard. Easy images are sparsely resampled with priority for long-unused ones, medium images are partially selected, and hard images are fully sampled in every epoch. This strategy periodically updates learning sufficiency, allowing the detector to focus on informative images and discard redundant ones. AFSS achieves over 1.43x training speedup for YOLO-series detectors on MS COCO 2017, PASCAL VOC 2007, DOTA-v1.0, and DIOR-R datasets, while also improving accuracy.

Key takeaway

For AI Engineers optimizing YOLO training pipelines, you should consider integrating dynamic sampling strategies like AFSS. This approach can significantly reduce training time by over 1.43x and enhance model accuracy by intelligently focusing on the most informative images, rather than exhaustively processing all data in every epoch. Evaluate your current training data for redundancy and implement a system to prioritize "hard" examples while still reviewing "easy" ones to prevent forgetting.

Key insights

Dynamically sampling training images based on learning sufficiency significantly accelerates YOLO training and improves accuracy.

Principles

• Prioritize hard examples for learning.
• Review easy examples to prevent forgetting.
• Periodically update learning sufficiency.

Method

AFSS measures learning sufficiency via minimum detection recall and precision, categorizing images into easy, medium, or hard for dynamic, adaptive sampling across epochs.

In practice

• Implement dynamic image sampling in object detection.
• Use recall/precision to gauge learning sufficiency.
• Prioritize recently unused images for review.

Topics

• YOLO Detectors
• Training Optimization
• Anti-Forgetting Sampling
• Object Detection
• Training Efficiency

Code references

• starriver030515/HAPO

Best for: Computer Vision Engineer, AI Engineer, AI Scientist, AI Researcher, Machine Learning Engineer, Deep Learning Engineer

Related on AIssential

• Less Data, Faster Training: repeating smaller datasets speeds up learning via sampling biases — Smaller datasets with repetition accelerate learning by inducing beneficial sampling biases that adjust layer-wise growth.
• The Optimizer in AI Training Is Not the Pilot — AI training needs a control layer to manage operational complexity, not just an optimizer for parameter updates.
• YOLO — You Only Look Once — A decade of real-time vision — YOLO transformed object detection into a single regression problem for real-time performance.
• Linearizing Vision Transformer with Test-Time Training — Linearizing Vision Transformers with Test-Time Training enables efficient weight transfer and faster inference.
• Preserving Plasticity in Continual Learning via Dynamical Isometry — Dynamical isometry is a key mechanism for preserving plasticity in continual learning.
• Experimental Analysis of Neural Network-Based Image Classification on the CIFAR-10 Dataset — Neural network image classification on CIFAR-10 reveals the practical distinction between representation learning and memorization.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.