Scaling up training dataset size for transcriptomic AI models is much pain with little gain

2026-06-09 · Source: Machine learning : nature.com subject feeds · Field: Science & Research — Artificial Intelligence & Machine Learning, Life Sciences & Biology, Health & Medical Research · Depth: Expert, quick

Summary

A study published in Nat. Methods on June 9, 2026, systematically evaluated the impact of training dataset size and diversity on single-cell foundation model performance. Researchers found that increasing dataset size beyond a specific saturation point offered "little gain" in model performance for transcriptomic AI models. This challenges the assumption that larger datasets, often tens of millions of cells, inherently lead to better outcomes for models like Geneformer and SCimilarity. The evaluation considered various single-cell foundation models and datasets, including the 22.2-million-cell scTab corpus, concluding that the advantages of scaling up training data for these models are limited. The findings suggest an optimal dataset size exists, beyond which the effort of scaling provides diminishing returns.

Key takeaway

For AI Scientists developing single-cell foundation models, recognize that scaling training datasets indefinitely offers diminishing returns. Your focus should shift from sheer volume to optimizing dataset diversity and quality, as performance gains plateau beyond a certain size. This insight can prevent wasted computational resources and time, guiding you to more efficient model development strategies for transcriptomic AI.

Key insights

Scaling single-cell foundation model training datasets beyond a saturation point yields minimal performance improvements.

Principles

• Transcriptomic AI models exhibit learning saturation.
• Dataset diversity is crucial for single-cell foundation models.

Method

Systematic evaluation of training dataset size and diversity on single-cell foundation model performance, using models like Geneformer and SCimilarity.

In practice

• Optimize dataset size for single-cell models.
• Prioritize dataset quality over quantity.

Topics

• Single-cell AI
• Foundation Models
• Transcriptomics
• Dataset Scaling
• Learning Saturation
• Geneformer

Best for: AI Scientist, Research Scientist

Related on AIssential

• Evaluating the role of pretraining dataset size and diversity on single-cell foundation model performance — Single-cell foundation models do not show clear data scaling laws, unlike large language models.
• Train-to-Test scaling explained: How to optimize your end-to-end AI compute budget for inference — Train-to-Test (T2) scaling laws optimize LLM compute across training and inference, favoring smaller, overtrained models for reasoning tasks.
• Scaling and quantization of a foundational deep learning model for network biology — Expanding Geneformer with more data and quantization improves biological prediction while reducing GPU demands.
• The next step towards AGI — AI progress is accelerating due to diverse innovations, shifting the focus from raw scale to sample efficiency and data compression.
• Less Data, Faster Training: repeating smaller datasets speeds up learning via sampling biases — Smaller datasets with repetition accelerate learning by inducing beneficial sampling biases that adjust layer-wise growth.
• ResNets of All Shapes and Sizes: Convergence of Training Dynamics in the Large-scale Limit — ResNet training dynamics converge to a quantifiable large-scale limit across depth, width, and embedding dimension.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine learning : nature.com subject feeds.