The next big breakthrough will be AIs learning on the job

2026-07-10 · Source: Dwarkesh Podcast · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, long

Summary

Current AI training, heavily reliant on Reinforcement Learning with Verifiable Rewards (RLVR) in simulated environments, faces limitations in generalizing to complex, real-world tasks like computer use or business building due to data inefficiency and the non-grindable nature of many domains. While proponents argue for scaling compute and in-context learning, a significant portion (30-50%) of lab compute is wasted on inference without improving models. The article argues for "continual learning," where models update their weights based on real-world deployment experiences. Proposed solutions include On-Policy Self-Distillation (OPSD), which distills session-learned knowledge into weights, and "dreaming," where AIs build and train against self-generated simulations. This shift is crucial for AIs to learn from scarce, unstructured real-world data and expand capabilities beyond initially trained, verifiable domains by 2027-2028.

Key takeaway

For AI Scientists and ML Engineers designing next-generation AI systems, recognize that current RLVR training alone will not yield broad real-world competence. You should prioritize developing continual learning mechanisms that update model weights from deployment experiences. Focus on techniques like On-Policy Self-Distillation or AI-driven "dreaming" to convert inference compute into valuable, on-the-job learning, ensuring your models can adapt to complex, non-grindable environments and expand their capabilities beyond initial training.

Key insights

Current AI training wastes valuable real-world data; continual learning is essential for true generalization beyond simulated environments.

Principles

• Real-world competence demands continual learning.
• Verifiable, grindable domains limit AI generalization.
• Inference compute offers untapped learning potential.

Method

On-Policy Self-Distillation (OPSD) distills session-learned knowledge into base model weights by matching predictions of an experienced "teacher" model. "Dreaming" involves AIs building and training against self-generated simulations.

In practice

• Capture deployment insights for model improvement.
• Apply OPSD to distill session knowledge into weights.
• Investigate AI-built simulations for skill rehearsal.

Topics

• Continual Learning
• On-Policy Self-Distillation
• Reinforcement Learning
• AI Generalization
• Sample Efficiency
• Inference Compute

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, Director of AI/ML

Related on AIssential

• Lessons from Trillion Token Deployments at Fortune 500s — Alessandro Cappelli, Adaptive ML — Reinforcement Learning systematically integrates feedback to accelerate LLM lifecycle, enabling production-ready, cost-effective, and performant models.
• Dertouzos Distinguished Lecturer: Richard Sutton — OAK proposes a domain-independent AI architecture that learns abstractions and skills from experience, scaling with computation.
• The Death of RLHF: A Practitioner’s Guide to the New Post-Training Stack — New post-training methods like GRPO, RLVR, and DAPO have replaced RLHF for reasoning tasks, improving scalability and efficiency.
• RL for Agents Workshop - Deep Dive on Training Agents with RL and Open Source — AI agents are evolving rapidly, enabling long-horizon tasks through advanced RL training, custom models, and robust evaluation.
• Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions — Bridging the RL research-to-deployment gap requires a cyclical statistical framework addressing data scarcity and environmental nonstationarity.
• Data News — Week 25.43 — Modern AI systems combine pre-training and reinforcement learning (RL) to achieve advanced reasoning and behavioral capabilities.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Dwarkesh Podcast.