The System Behind Self-Driving: Waymo’s Dmitri Dolgov

2026-04-17 · Source: The a16z Show · Field: Technology & Digital — Robotics & Autonomous Systems, Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Intermediate, extended

Summary

Waymo currently provides over 500,000 fully autonomous rides weekly across 11 U.S. cities, utilizing approximately 3,000 vehicles and covering over four million autonomous miles. This achievement stems from a comprehensive system for training, evaluating, and deploying its AI driver, which integrates sensor fusion from LiDAR, radar, and cameras. The architecture involves a large off-board foundation model, specialized into three "teacher" models (Waymo driver, simulator, critic), then distilled for on-board inference. Waymo emphasizes that full autonomy fundamentally differs from driver-assist systems. The company is now focused on accelerated global scaling, with plans to launch in London and Tokyo this year, leveraging rapid AI advancements and a new, lower-cost sixth-generation hardware and software stack.

Key takeaway

For AI and MLOps Engineers developing autonomous systems, recognize that achieving full autonomy, as demonstrated by Waymo's 500,000 weekly rides, necessitates a complete ecosystem beyond core models. Prioritize investing in robust training, simulation, and evaluation infrastructure, including "critic" models and multi-modal sensor fusion. This holistic approach, distinct from driver-assist, is crucial for scaling deployments globally and reducing operational costs.

Key insights

Full autonomy demands a comprehensive AI ecosystem, integrating advanced sensing, simulation, and specialized models for robust, scalable deployment.

Principles

• Full autonomy is qualitatively distinct from driver-assist.
• Sensor fusion across LiDAR, radar, cameras is critical.
• Iterative learning and architectural evolution drive progress.

Method

Waymo's method involves a large off-board foundation model, specialized into "teacher" models (driver, simulator, critic), then distilled for on-board inference, augmented with structured intermediate representations for robust simulation and safety.

In practice

• Combine LiDAR, radar, and cameras for robust perception.
• Utilize realistic simulators for closed-loop training.
• Employ critic models to define reward functions for RLFT.

Topics

• Waymo
• Autonomous Driving
• Sensor Fusion
• Foundation Models
• Simulation
• MLOps

Best for: AI Engineer, MLOps Engineer, Director of AI/ML

Related on AIssential

• Can autonomous AI-powered killer drones take morality onboard? | Drones (military) — The central question is whether autonomous AI weapons can incorporate human morality for ethical combat decisions.
• Why Video Agent models are next — Ethan He, xAI Grok Imagine — Language models are becoming the primary drivers of intelligence and capability in advanced video generation and world models.
• ACM CAIS: Conference on AI and Agentic Systems — Formal verification of multi-agent coordination protocols significantly enhances system reliability and performance by preventing resource conflicts.
• RoboDream: Compositional World Models for Scalable Robot Data Synthesis — A generalizable embodiment-centric world model synthesizes photorealistic robot demonstrations by decoupling motion from environment synthesis.
• OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agents — Online multi-turn reinforcement learning on live websites offers a scalable path for training capable visual web agents.
• The Lie We Tell: Correcting the Euclidean Fallacy in Vision Language Action Policies via Score Matching on Tangent Space — Lie Diffuser Actor corrects geometric errors in VLA policies by operating intrinsically on the SE(3) manifold, ensuring geodesic optimality.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The a16z Show.