PerceptDrive: Perception Prior World-Action Modeling with Adaptive Expert Routing for End-to-End Autonomous Driving

2026-07-23 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Emerging Technologies & Innovation · Depth: Expert, extended

Summary

PerceptDrive is a novel perception prior world-action modeling framework designed for end-to-end autonomous driving, achieving state-of-the-art performance with 90.4 PDMS on NAVSIM v1 and 90.2 EPDMS on NAVSIM v2. This framework addresses the "prior-to-plan transfer" problem by integrating teacher-distilled geometric, semantic, and dynamic priors from a frozen, driving-adapted perception provider and dense observation latents from a frozen self-supervised video encoder. It employs a trainable expert-routed world-action model with per-branch prior retention to preserve source-specific information and a scene-conditioned router that adaptively combines expert conditions. During inference, PerceptDrive generates a single trajectory per planning step using one front-facing camera, without requiring test-time scoring, reranking, or search. The system comprises 421M trainable parameters and costs approximately 409.6 MI308X GPU-hours for training.

Key takeaway

For machine learning engineers developing end-to-end autonomous driving systems, consider integrating PerceptDrive's approach to enhance planning performance and efficiency. By anchoring expert readouts with prior retention and adaptively routing them based on scene context, you can achieve state-of-the-art results without complex test-time candidate selection. This method streamlines inference, making your models more practical for real-world deployment.

Key insights

PerceptDrive improves autonomous driving by adaptively routing and retaining perception priors for direct trajectory generation.

Principles

• Preserve source-specific information via prior retention.
• Dynamically adapt expert contributions per scene.
• Distill privileged metrics for efficient routing.

Method

PerceptDrive constructs a frozen perception provider, then trains an expert-routed world-action model. This model uses query banks, per-branch prior retention, and a scene-conditioned router to combine expert conditions before flow-matching trajectory generation.

In practice

• Implement per-branch retention for specialized readouts.
• Use scene-conditioned routing for dynamic expert fusion.
• Distill privileged metrics to optimize inference gates.

Topics

• End-to-End Autonomous Driving
• Perception Priors
• World-Action Models
• Adaptive Expert Routing
• Trajectory Generation
• NAVSIM Benchmark

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• PerceptDrive: Perception Prior World-Action Modeling with Adaptive Expert Routing for End-to-End Autonomous Driving — PerceptDrive improves direct planning by adaptively routing and preserving perception priors from frozen foundation models.
• PersonaDrive: Human-Style Retrieval-Augmented VLA Agents for Closed-Loop Driving Simulation — PersonaDrive enables diverse, human-style driving agents in simulations by conditioning VLAs on retrieved style-instructed human demonstrations.
• WorkDrive: Roadwork Chain of Causation for Autonomous Driving — WorkDrive improves autonomous driving in roadwork zones by integrating perception-grounded causal reasoning with trajectory prediction.
• CLEAR: Cognition and Latent Evaluation for Adaptive Routing in End-to-End Autonomous Driving — CLEAR achieves real-time, multi-modal autonomous driving by replacing iterative denoising with single-step latent space drift and semantic reasoning.
• New driving AI ranks possible routes for safer, clearer decisions — SafeDrive is an E2E AI model that ranks routes for safer autonomous driving decisions.
• Driving the Wrong Way: Leveraging Interpretability in End2End Autonomous Driving Models — Interpretable latent concepts in end-to-end autonomous driving models enable targeted interventions to improve safety and performance.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.