"How to measure intelligence?" | Six researchers debate

2025-11-03 · Source: ARC Prize · Field: Technology & Digital — Artificial Intelligence & Machine Learning, AI & Cognitive Science · Depth: Expert, extended

Summary

A panel discussion explored the fundamental differences between human and artificial intelligence, particularly focusing on the "anti-developmental" nature of current AI systems. Laura, a developmental researcher, highlighted that while AI can solve complex problems like mathematical proofs in seconds, it struggles with simple common-sense tasks, lacking a progressive learning path akin to human development. The debate extended to defining intelligence, with some panelists emphasizing efficient generalization across tasks, while others, like Laura, pointed to the role of open-ended play, curiosity, and intrinsic goal-setting in human cognition. The discussion also touched upon the utility and limitations of benchmarks like ARC AGI in measuring human-like intelligence, with a consensus that while benchmarks drive progress, they often fail to capture the richness of human thinking, problem-making, and the reward of generating new, even "wrong," ideas. The role of world models in language models and the potential of video games to unlock new AI capabilities were also discussed.

Key takeaway

For AI scientists and research scientists aiming to bridge the gap between current AI and human-like intelligence, you should critically evaluate existing benchmarks. Consider expanding your research to include metrics for open-ended problem-making, intrinsic goal-setting, and the ability to generate novel ideas, even those initially deemed "wrong." This shift will help move beyond mere task efficiency to capture the richer, developmental aspects of human cognition, fostering AI systems that can truly learn and adapt in complex, unstructured environments.

Key insights

Current AI lacks human-like developmental learning, struggling with common sense despite advanced problem-solving.

Principles

• Human intelligence progresses developmentally.
• Play and curiosity are central to human learning.
• Benchmarks can limit AI progress by oversimplifying intelligence.

Method

The discussion implicitly contrasts current AI's data-intensive, task-specific learning with human intelligence's open-ended, curiosity-driven exploration and goal-setting, suggesting a need for AI to emulate developmental learning paths.

In practice

• Focus AI research on developmental learning paths.
• Design benchmarks for open-ended problem-making.
• Investigate AI's ability to set its own objective functions.

Topics

• Measuring Intelligence
• Human-like AI
• Child Development Research
• AI Benchmarks
• Play and Exploration

Best for: AI Scientist, Research Scientist

Related on AIssential

• [AINews] Humanity's Last Gasp — AI's progress in automation paradoxically correlates with increased human workload, highlighting a gap in generalizable learning.
• Why comparisons between AI and human intelligence miss the point — Human intelligence is fundamentally collective, embodied, and social, a dimension AI systems currently lack.
• AGI: What Gets Measured Gets Built — Current AGI evaluation methods are flawed, shaping AI to pass tests rather than exhibit true, safe intelligence.
• Measuring progress toward AGI: A cognitive framework — A cognitive taxonomy and evaluation protocol are proposed to empirically measure AGI progress against human capabilities.
• What the Freakiness of 2025 in AI Tells Us About 2026 — AI progress in 2025 highlighted both advanced reasoning and generative capabilities, alongside challenges in trust and general intelligence.
• On Games -- and Play | ARC Prize @ MIT — Human intelligence is uniquely defined by the capacity to invent arbitrary problems and objective functions, not just efficient skill acquisition.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ARC Prize.