Automatic Combination of Sample Selection Strategies for Few-Shot Learning

2026-04-21 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

A new method called Automatic Combination of Sample Selection Strategies (ACSESS) has been developed to enhance few-shot learning performance by combining individual sample selection strategies. Researchers investigated 20 existing sample selection strategies across 5 few-shot learning approaches and 8 image and 6 text datasets. They found that while individual strategies often fail to consistently outperform random selection, ACSESS consistently improves performance, achieving up to a 1.8 percentage point increase for gradient few-shot learning and 2.5 percentage points for in-context learning. The study also revealed that sample selection is most impactful at lower shot counts, with its benefit diminishing and regressing to random selection at higher numbers of shots (30-40 shots). Furthermore, increasing shots beyond a certain point (50 for gradient, 20-25 for in-context) provides negligible or even negative performance gains.

Key takeaway

Research Scientists working with few-shot learning models should integrate advanced sample selection methods like ACSESS, particularly for tasks with very limited labeled data (e.g., 1-shot to 5-shot scenarios). Focusing on samples with high learnability and leveraging strategy combinations can yield significant accuracy improvements. Be aware that increasing the number of shots beyond 20-40 may not provide further benefits and can even degrade performance for in-context learning due to context size limitations.

Key insights

Combining diverse sample selection strategies significantly boosts few-shot learning, especially at low shot counts.

Principles

• Learnability is a strong indicator of sample quality.
• Sample selection impact is modality and approach dependent.
• Benefit of selection diminishes at higher shot counts.

Method

ACSESS identifies relevant strategies via forward, backward, or Datamodels selection, then combines them using weighted averaging of normalized sample scores.

In practice

• Prioritize learnability-focused strategies for few-shot tasks.
• Use ACSESS for improved performance in few-shot settings.
• Avoid excessive shots; performance plateaus or declines.

Topics

• Few-Shot Learning
• Sample Selection Strategies
• ACSESS Method
• In-Context Learning
• Meta-Learning

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Re-Evaluating Continual Learning with Few-Shot Adaptation — Few-shot evaluation offers a more comprehensive assessment of continual learning model stability and plasticity than traditional 0-shot methods.
• Data Presentation Over Architecture: Resampling Strategies for Credit Risk Prediction with Tabular Foundation Models — Context construction significantly impacts Tabular Foundation Model performance in imbalanced credit risk prediction.
• LOTTERY: Learning from Reference-Only Samples in Two-Sample Testing under Size Asymmetry — LOTTERY enables robust two-sample testing in few-shot, imbalanced settings by learning reference-dependent representations from abundant reference data.
• Provable Data Scaling Law for Meta Learning via Complexity Minimization — Complexity minimization provides a theoretical framework for data scaling laws in meta-learning, improving few-shot adaptation with more meta-training data.
• 1S-DAug: One-Shot Data Augmentation for Robust Few-Shot Generalization — 1S-DAug improves few-shot learning generalization by generating diverse, faithful image variants at test time via diffusion.
• How Useful is Causal Invariance for Domain Adaptation in Finite-Sample Settings? — Causal invariance improves supervised domain adaptation in finite-sample settings when target-risk margins between candidate models are sufficiently large.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.