AdaStop: Cost-Aware Early Stopping for DNN Test Selection

2026-07-06 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

AdaStop is a novel framework designed for cost-aware early stopping in deep neural network (DNN) testing, addressing the challenge of optimizing labeling budgets. Traditional DNN testing methods focus on identifying faults under a predetermined budget, often leading to either insufficient testing or excessive labeling costs. AdaStop reframes DNN testing as a cost-benefit decision process, where labeling an input incurs a cost $c$ and discovering a fault yields a value $v$. The framework operates by continuously estimating the marginal fault discovery rate during testing and halts the labeling process when this rate drops below a predefined threshold, $τ= c/v$. Experimental results demonstrate AdaStop's efficiency, enabling the discovery of 65-84% of faults while utilizing only 9-31% of the total labeling budget across diverse datasets, network architectures, and test selection strategies.

Key takeaway

For Machine Learning Engineers managing DNN testing and labeling budgets, AdaStop offers a critical shift from fixed budgets to dynamic, cost-aware stopping. You should consider integrating a framework like AdaStop to optimize resource allocation, potentially discovering 65-84% of faults with only 9-31% of traditional labeling costs. This approach ensures more efficient quality assurance by halting testing precisely when marginal fault discovery no longer justifies the labeling expense.

Key insights

AdaStop optimizes DNN testing costs by dynamically stopping labeling when the marginal fault discovery rate falls below a cost-benefit threshold $τ= c/v$.

Principles

• DNN testing is a cost-benefit decision.
• Dynamic stopping optimizes labeling budgets.
• Marginal fault discovery rate is a stopping signal.

Method

AdaStop estimates the marginal fault discovery rate during DNN testing. It stops labeling when this rate falls below the threshold $τ= c/v$, balancing labeling cost $c$ against fault discovery value $v$.

In practice

• Reduce DNN test labeling costs.
• Improve fault discovery efficiency.
• Implement $τ= c/v$ as a stopping criterion.

Topics

• AdaStop
• DNN Testing
• Early Stopping
• Cost Optimization
• Fault Discovery
• Machine Learning

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Models — Learned stopping rules improve reasoning model efficiency when scalar signals are insufficient, but their value is task-dependent.
• Stop the Sampler! Classifier-Based Adaptive Stopping for Sampling Kernels — Adaptive MCMC stopping can be learned via GFlowNet-trained neural classifiers, dynamically optimizing trajectory lengths for efficiency.
• Evaluation-Strategy Gap in Fault Diagnosis of Deep Learning Programs — Fault diagnosis in DL programs exhibits an evaluation gap when applied to unseen code, driven by feature structure.
• ILDR: Geometric Early Detection of Grokking — ILDR provides an early, robust geometric signal for grokking, enabling earlier intervention and reduced training.
• Regret-Based $(ε,δ)$-optimal Stopping Criteria for Bayesian Optimization — New regret bounds enable provably optimal stopping for Bayesian Optimization, ensuring ε-optimality with high probability.
• Filtered ANN as a Phase Transition: When Selectivity-Estimation Error Causes Plan Regret — Selectivity-estimation error causes significant plan regret in filtered ANN queries, especially near strategy phase boundaries.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.