Why hasn't longer-horizon training slowed AI progress?

2026-05-06 · Source: All posts - seangoedecke.com RSS feed · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Intermediate, medium

Summary

This analysis addresses Dwarkesh Patel's question regarding why AI progress has not slowed down more, despite the expectation that training more powerful models for harder tasks would demand significantly more FLOPs and time. The author proposes three main reasons. Firstly, AI labs are achieving orders of magnitude more efficiency from existing FLOPs, often by fixing "boneheaded mistakes" like the FP16 issue in early GPT-4 training. Secondly, human intuition about AI progress is unreliable; it's easier to detect when an AI is dumber than when it's smarter, making the "real" rate of intelligence growth hard to gauge. Thirdly, AI model capabilities stem from a constellation of traits beyond raw intelligence, such as working memory, tool familiarity, or persistence, which can be improved through clever tricks or prompt tweaks rather than just brute-force FLOPs.

Key takeaway

For AI Scientists evaluating model progress, recognize that perceived slowdowns might be due to human judgment biases or efficiency gains. Focus your efforts on identifying and resolving "boneheaded mistakes" in training pipelines, as these yield significant FLOPs efficiency. Also, explore prompt engineering and system tweaks to enhance traits like persistence or working memory, which can dramatically improve model capabilities beyond raw intelligence.

Key insights

AI progress persists due to efficiency gains, unreliable human perception, and capabilities driven by diverse traits beyond raw intelligence.

Principles

• AI efficiency is often limited by bugs, not genius.
• Human judgment of AI intelligence is inherently biased.
• Model capabilities arise from multiple, distinct traits.

In practice

• Debugging training code significantly boosts FLOPs efficiency.
• System prompts can enhance non-intelligence traits.
• Consider "persistence" when evaluating model reasoning.

Topics

• AI Progress
• FLOPs Efficiency
• Model Capabilities
• AI Evaluation
• Prompt Engineering
• Agentic AI

Best for: AI Scientist, Research Scientist, Director of AI/ML

Related on AIssential

• AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks? — Long-horizon AI agent success hinges on persistent iteration and empirical feedback, not just initial solution quality.
• AI keeps getting smarter, so why does it still fail at obvious things? — AI's apparent intelligence stems from pattern matching, not true comprehension, leading to reliability gaps.
• Breaking the Spell of Vibe Coding — Vibe coding can induce a "dark flow" state, similar to gambling, leading to misperceptions of productivity and skill development.
• LAI #128: Developers Using AI Were 19% Slower. They Thought They Were 20% Faster. — The perception of AI's impact on developer productivity often misaligns with actual performance, especially on complex tasks.
• The AI Progress Chart Everyone Is Misreading — Beth Barnes & David Rein — AI evaluation requires unified metrics like human task completion time to track progress and understand capabilities across diverse models.
• Slow down to speed up: so much has changed in 6 months’ time — Rapid AI agent adoption boosts individual dev output but risks software quality and organizational integrity without human oversight.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by All posts - seangoedecke.com RSS feed.