On the slow death of Scaling (birth of Adaption Labs) | Sara Hooker | HF ML Club India EP2

2026-05-21 · Source: HuggingFace · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation, Software Development & Engineering · Depth: Advanced, extended

Summary

Dr. Sara Hooker, co-founder of Adaption Labs, argues that the era of monolithic AI scaling, characterized by a "bigger is better" approach and the "slow death of scaling" paper, is ending. She highlights evidence contradicting this view, including small models outperforming large ones, severe weight redundancies, and disappointing performance gains from recent large models like GPT-4.5, Llama 4, and Mythos, which were often too expensive to serve. Hooker advocates for a new era of "adaptive intelligence," focusing on post-training, test-time scaling, and continuous learning. Adaption Labs champions optimizing in the data space, exemplified by their "adaptive data" release, and automating end-to-end fine-tuning through "auto scientists" to achieve real-time adaptation and efficiency, shifting the focus from pre-training compute to dynamic, test-time compute.

Key takeaway

For ML Engineers designing next-generation AI systems, recognize the diminishing returns of brute-force model scaling. Shift your focus to adaptive intelligence, prioritizing efficient post-training, test-time scaling, and continuous learning. Invest in strategies like data space optimization and automated fine-tuning to build more flexible, cost-effective, and responsive models that adapt in real-time, ensuring your solutions remain competitive and relevant.

Key insights

Monolithic AI scaling is ending; the future is adaptive, efficient, post-training intelligence.

Principles

• Scaling compute for model size yields decreasing returns.
• Efficiency drives adaptation and learning speed.
• Optimize in the data space for targeted model behavior.

Method

Automate end-to-end fine-tuning via "auto scientists" to optimize data, learn from tasks, and adapt real-time, co-designing models with serving.

In practice

• Prioritize test-time compute for high-uncertainty examples.
• Inject new data at different training stages (pre-training, post-training, RHF).
• Leverage adaptive data to target missing distribution parts.

Topics

• Adaptive Intelligence
• Model Scaling
• Machine Learning Efficiency
• Test-Time Training
• Data Curation
• Continual Learning

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Architect

Related on AIssential

• From bigger models to better intelligence: what NeurIPS 2025 tells us about progress — AI progress is shifting from raw scale to efficiency, real-world evaluation, and interactive world modeling.
• Scalable Reinforcement Learning via Adaptive Batch Scaling — RL non-stationarity evolves, requiring adaptive batch scaling for optimal performance and model scaling.
• Learning, Fast and Slow: Towards LLMs That Adapt Continually — Combining slow parameter updates with fast in-context learning improves LLM adaptation and reduces forgetting.
• How do you experiment with a (very) large model architecture? [D] — Scaling laws and hyperparameter transfer schemes are crucial for efficient experimentation with large, compute-intensive models.
• The next step towards AGI — AI progress is accelerating due to diverse innovations, shifting the focus from raw scale to sample efficiency and data compression.
• AI Isn’t Hitting a Scaling Wall. It’s Hitting a Measurement Wall. — AI's efficiency gap and "emergent" behaviors stem from a "measurement wall," not a scaling limit, due to evaluation methods collapsing complex internal states.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by HuggingFace.