Learning to Theorize the World from Observation

2026-06-06 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, long

Summary

The Learning-to-Theorize (L2T) paradigm introduces a novel approach to AI understanding, moving beyond mere future prediction to infer explicit explanatory theories from raw, non-textual observations. Inspired by developmental cognitive science, this framework posits that true understanding emerges from constructing internal theories of how the world operates. The Neural Theorizer (NEO) instantiates L2T as a probabilistic neural model that induces latent, executable, and compositional programs, representing a learned "Language of Thought." NEO executes these programs through a shared transition model, enabling explanation-driven generalization. Experiments demonstrate its ability to transfer inferred programs, recombine primitives into unseen compositions, and generalize to programs longer than those observed during training, addressing limitations of traditional world models.

Key takeaway

For AI and Research Scientists developing robust world models, this work suggests shifting focus from optimizing future prediction to inducing explicit, compositional theories. Your systems should aim to discover reusable abstract primitives and learn how to compose them into structured explanations. Implementing mechanisms like NEO's latent program induction, shared execution model, and Minimum Description Length principle can significantly improve compositional and length generalization, crucial for understanding novel phenomena and out-of-distribution scenarios.

Key insights

Understanding the world means inferring explicit, compositional programs from raw observations, not just predicting future states.

Principles

• Human-like understanding requires theory-building.
• Theories are executable, compositional programs.
• Learning should target reusable explanatory structures.

Method

NEO infers latent programs by iteratively selecting primitives via a goal-conditioned policy and executing them through a shared transition model, optimized with an MDL principle for program length.

In practice

• Train on minimally curated observational data.
• Evaluate by program transferability, not just reconstruction.
• Use state grounding to enforce valid intermediate states.

Topics

• Learning-to-Theorize
• Neural Theorizer
• Program Induction
• Compositional Generalization
• World Models
• Language of Thought

Best for: AI Scientist, Research Scientist

Related on AIssential

• AGEL-Comp: A Neuro-Symbolic Framework for Compositional Generalization in Interactive Agents — AGEL-Comp combines neuro-symbolic AI to enhance compositional generalization in interactive agents.
• Structural Generalization on SLOG without Hand-Written Rules — A neural cellular automaton can achieve structural generalization in semantic parsing without hand-written rules.
• What do Language Models Learn and When? The Implicit Curriculum Hypothesis — LLM pretraining follows a consistent, compositional skill emergence order, predictable from internal representations.
• Roundtables: Can AI Learn to Understand the World? — World models are emerging as a critical AI frontier to ground systems in real-world understanding, addressing LLM limitations in common sense and causality.
• Toward Compiler World Models: Learning Latent Dynamics for Efficient Tensor Program Search — Modeling tensor program optimization as latent dynamics improves efficiency and performance over static evaluation.
• There Will Be a Scientific Theory of Deep Learning — A scientific theory of deep learning, "learning mechanics," is emerging, focusing on dynamics, aggregate statistics, and quantitative predictions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.