Jul 9, 2026Frontier Red TeamClaude plays robotics

2026-07-09 · Source: Anthropic Research · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Advanced, extended

Summary

Anthropic's research, published July 9, 2026, investigates the transferability of language models' strengths to robotics, focusing on scene perception, robot state understanding, and physical action. The study tested models like Claude Opus 4.6, Mythos Preview, GPT-5.4, and Gemini 3.1 across various robot bodies, including a simulated quadruped, humanoid, robotic arm, and a real Unitree Go2. Models were evaluated using diverse control interfaces, from direct motor torques to high-level pretrained policies, in classic control, locomotion, and manipulation tasks. Findings indicate that while models struggle with direct joint control, their capabilities significantly improve when supervising pretrained controllers or using simple orientation tools. Newer models show substantial gains in adjusting strategies and converting sensory understanding into actions, with Mythos Preview achieving up to 5.5% manipulation success and Opus 4.6 balancing a Go2 robot for nearly two seconds.

Key takeaway

For robotics engineers developing AI-driven systems, prioritize integrating language models with high-level pretrained policies rather than direct low-level control. Your capability assessments should account for the control interface, as small tool changes significantly alter performance. Consider using LLMs to supervise existing controllers or generate training data, but be aware of their current limitations in stable spatial memory and long-range planning. This approach will maximize current LLM utility in physical world applications.

Key insights

Language models excel in robotics when supervising high-level policies, not direct low-level control.

Principles

• Control interface heavily dictates LLM robotics capability.
• Generational LLM improvements are uneven across control types.
• Spatial memory and long open-loop plans remain LLM weaknesses.

Method

Evaluated LLMs on classic control, locomotion, and manipulation tasks using direct, programmatic, policy, and reinforcement learning supervision interfaces across diverse robot bodies in simulation and real-world.

In practice

• Integrate LLMs with pretrained robot policies for best results.
• Provide simple orientation tools like a compass for navigation.
• Use LLMs to generate training data for robotic systems.

Topics

• Language Models
• Robotics Control
• Humanoid Robots
• Quadruped Robots
• Robot Manipulation
• Vision-Language-Action
• Reinforcement Learning

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• LLM-Powered Interactive Robotic Action Synthesis from Multimodal Speech, Gestures, and Music — LLMs can synthesize complex robot actions from multimodal human inputs like speech, gestures, and music.
• From Digital to Physical: Digital Agents as Autonomous Coaches for Physical Intelligence — LLM agents can autonomously engineer embodied policies, outperforming human baselines and self-correcting through iterative environment feedback.
• Why Robots Still Struggle With Simple Tasks (And What Might Finally Change That) | Karol Hausman, Co-Founder & CEO of Physical Intelligence — Combining internet-scale LLM knowledge with robot motion data enables general physical intelligence for complex tasks.
• LLM-Foraging: Large Language Models for Decentralized Swarm Robot Foraging — Integrating LLMs at key decision points in swarm robotics enhances adaptability and performance without costly re-optimization.
• LLMs help robots understand vague instructions and focus on key details — Masked IRL uses dual LLMs to clarify vague robot instructions and filter environmental noise, reducing demonstration data needs.
• AI-IoT-Robotics Integration: Survey of Frameworks, Emerging Trends, and the Path Toward Connected Robotics — Unified AI-IoT-Robotics frameworks are crucial for real-time, intelligent systems, utilizing SLMs at the edge and LLMs in the cloud.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Anthropic Research.