PerceptDrive: Perception Prior World-Action Modeling with Adaptive Expert Routing for End-to-End Autonomous Driving

2026-07-22 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Computer Vision · Depth: Expert, quick

Summary

PerceptDrive is a novel perception prior world-action modeling framework designed for end-to-end autonomous driving, featuring adaptive expert routing. It integrates teacher-distilled priors from a frozen, driving-adapted provider and dense observation latents from a frozen self-supervised video encoder into a trainable expert-routed world-action model. The system employs expert-specific query branches, a prior-retention objective to anchor each branch, and a router that predicts soft gates from a shared scene representation to combine expert conditions for trajectory generation. During training, privileged rule-based sub-metric estimates guide soft-gate distillation. At inference, PerceptDrive operates with a single front-facing camera, generating one trajectory per planning step without test-time scoring or search. It achieves state-of-the-art performance, with 90.4 PDMS on NAVSIM v1 and 90.2 EPDMS on NAVSIM v2. Ablations confirm the benefits of prior retention and scene-conditioned routing, alongside differential reliance on its three priors.

Key takeaway

For Machine Learning Engineers developing end-to-end autonomous driving systems, PerceptDrive demonstrates a critical shift. You should consider integrating frozen perception foundation models with adaptive expert routing to enhance planning accuracy and efficiency. This approach, which avoids complex test-time candidate selection, can significantly improve performance metrics like PDMS and EPDMS, streamlining your inference pipeline and leveraging rich prior knowledge effectively.

Key insights

PerceptDrive improves direct planning by adaptively routing and preserving perception priors from frozen foundation models.

Principles

• Frozen perception models offer rich driving knowledge.
• Adaptive routing enhances task-relevant cue utilization.
• Prior retention anchors expert branch contributions.

Method

PerceptDrive feeds teacher-distilled priors and dense observation latents into an expert-routed world-action model, using a router for soft-gate prediction and a flow-matching actor for trajectory generation.

In practice

• Integrate frozen foundation models for driving tasks.
• Implement adaptive routing for scene-specific expert fusion.
• Design prior-retention objectives for model stability.

Topics

• End-to-End Autonomous Driving
• Perception Priors
• Adaptive Expert Routing
• World-Action Modeling
• Flow-Matching Actor
• NAVSIM Benchmarks

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• PerceptDrive: Perception Prior World-Action Modeling with Adaptive Expert Routing for End-to-End Autonomous Driving — PerceptDrive improves autonomous driving by adaptively routing and retaining perception priors for direct trajectory generation.
• EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Driving — Vision-centric foundation models exhibit a structural deficit in grounding ego-motion understanding in visual observations, relying instead on language modality.
• CLEAR: Cognition and Latent Evaluation for Adaptive Routing in End-to-End Autonomous Driving — CLEAR achieves efficient, multi-modal autonomous driving by combining single-step generative planning with LLM-driven cognitive reasoning.
• Perceive Before Reasoning: A Pre-Reasoning Perception Framework for Efficient and Reliable Proactive Mobile Agents — Perceiving before reasoning significantly improves proactive mobile agent efficiency and reliability by gating interventions.
• New driving AI ranks possible routes for safer, clearer decisions — SafeDrive is an E2E AI model that ranks routes for safer autonomous driving decisions.
• Driving the Wrong Way: Leveraging Interpretability in End2End Autonomous Driving Models — Interpretable latent concepts in end-to-end autonomous driving models enable targeted interventions to improve safety and performance.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.