If You Can't See Inside, How Do You Know It's THINKING? [Dr. Jeff Beck]

2026-01-25 · Source: Machine Learning Street Talk · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Emerging Technologies & Innovation · Depth: Advanced, extended

Summary

Dr. Jeff Beck and an interviewer discuss various advanced AI concepts, beginning with geometric deep learning's role in modeling the physical world by incorporating symmetries like translation and rotation invariance. The conversation then shifts to defining and measuring "agency" in AI, distinguishing between a simple computational agent and one exhibiting planning, counterfactual reasoning, and goal-oriented behavior. They explore the challenge of identifying true planning from sophisticated policy execution, suggesting that an agent requires physical embodiment and a degree of internal sophistication. The discussion transitions to energy-based models (EBMs), highlighting their advantage over traditional function approximation by incorporating inductive priors and internal state optimization, with Variational Autoencoders (VAEs) cited as a canonical example. They also touch upon test-time training, transduction, and the relationship between EBMs and Bayesian inference, including Joint Embedding Prediction Architectures (JEPAs) and self-supervised learning's role in maintaining rich representations.

Key takeaway

For AI researchers and scientists developing advanced models, consider integrating geometric deep learning principles to leverage physical symmetries, which can enhance model robustness. When designing intelligent agents, focus on architectures that enable explicit planning and counterfactual reasoning, rather than merely sophisticated policy execution, and explore energy-based models for their inherent inductive priors and dual optimization of internal states and outputs. Your approach to defining and measuring agency should account for internal computational sophistication and potential physical embodiment.

Key insights

Defining AI agency requires observing internal planning, not just sophisticated external behavior, and often implies physical embodiment.

Principles

• Models of the physical world should incorporate existing symmetries.
• Agency is a matter of degree, not a binary state.
• Science prioritizes prediction and data compression.

Method

Energy-based models (EBMs) optimize both prediction error and internal states, effectively placing constraints on input-output relationships, often exemplified by VAEs that regularize internal representations.

In practice

• Pre-process neural datasets with PCA and VAEs to assess signal-to-noise.
• Use self-supervised learning to avoid discarding task-irrelevant information.

Topics

• AI Agency
• Geometric Deep Learning
• Energy-Based Models
• Self-Supervised Learning
• AI Safety

Best for: AI Researcher, AI Scientist, Research Scientist

Related on AIssential

• Shape, Symmetries, and Structure: The Changing Role of Mathematics in Machine Learning Research — Mathematics' role in ML is shifting from theoretical guarantees to post-hoc explanations and high-level architectural guidance, leveraging advanced pure math.
• I Patented a Four-Sided Box. It's the Best Mental Model I Have for Building Agents. — Effective AI agents depend more on problem representation than on model size, a lesson from a patented "four-sided box" method.
• The Mathematical Foundations of Intelligence [Professor Yi Ma] — Intelligence can be formalized through parsimony and self-consistency, enabling principled AI architecture design.
• AI Isn’t Hitting a Scaling Wall. It’s Hitting a Measurement Wall. — AI's efficiency gap and "emergent" behaviors stem from a "measurement wall," not a scaling limit, due to evaluation methods collapsing complex internal states.
• “It’s the Geometry, Stupid”: How Isomorphic Labs Is Changing the Whole AI Industry — Isomorphic Labs is pioneering AI architectures based on diverse geometries, moving beyond current scale-focused models.
• Three ways AI is learning to understand the physical world — World models address LLM limitations in physical causality by providing AI with internal simulation capabilities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning Street Talk.