Cognitive Dual-Process Planning for Autonomous Driving with Structured Scene Knowledge and Verifiable Reasoning-Action Consistency

2026-07-23 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

A cognitive dual-process planning framework for autonomous driving integrates structured scene knowledge and adaptive reasoning. This framework uses a machine-parsable structured chain-of-thought (S-CoT) schema to represent planning-relevant scene knowledge. An automated data engine, leveraging perception foundation models and an expert VLM (Qwen3-VL-235B-A22B), generates S-CoT supervision without manual annotation. A lightweight visual Arbiter dynamically routes inputs to either fast meta-action prediction or slow structured reasoning based on estimated scene complexity. For slow-path outputs, a deterministic rule-based validator ensures consistency between parsed S-CoT fields and the final meta-action, providing verifiable rewards for Group Relative Policy Optimization (GRPO). In a 195-scene audit, annotations achieved 91.8% CoT accuracy and 98.5% Logical Consistency Score (LCS). On 574 NAVSIM test samples, the planner achieved 80.14% planning accuracy and 97.20% LCS, reducing average latency by 17.39% compared to static slow reasoning. Performance degraded in low-visibility and traffic-sign reasoning long-tail scenarios.

Key takeaway

For machine learning engineers developing autonomous driving systems, integrating a dual-process VLM architecture can significantly enhance planning accuracy and efficiency. You should consider implementing a visual arbiter to dynamically route scenes, reducing latency by 17.39% compared to static slow reasoning. Additionally, using rule-based verification for structured chain-of-thought rationales ensures 97.20% reasoning-action consistency, crucial for safety-critical applications. Be aware that performance may degrade in low-visibility or complex traffic-sign scenarios.

Key insights

Adaptive dual-process VLM planning with structured knowledge and verifiable consistency improves autonomous driving performance and efficiency.

Principles

• Explicit scene knowledge improves VLM planning traceability.
• Adaptive reasoning reduces latency in routine driving.
• Rule-based verification enforces reasoning-action consistency.

Method

The framework uses Qwen3-VL as a VLM backbone. An automated data engine generates S-CoT supervision. A visual Arbiter routes inputs. Supervised Fine-Tuning (SFT) establishes output modes, followed by Reinforcement Learning with Verifiable Rewards (RLVR) using GRPO for consistency.

In practice

• Implement a visual arbiter for dynamic VLM reasoning.
• Use rule-based validators for VLM planning consistency.
• Generate structured CoT supervision automatically.

Topics

• Autonomous Driving
• Vision-Language Models
• Dual-Process Planning
• Structured Chain-of-Thought (S-CoT)
• Reinforcement Learning (RLVR, GRPO)
• Scene Complexity Routing
• Verifiable Reasoning

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints — LVLMs can achieve robust, consistent visual reasoning by training with logically equivalent question pairs and a dual accuracy-consistency reward.
• Learning Visual Spatial Planning from Symbolic State via Modality-Gap-Aware Self-Distillation — Bridging the perception-reasoning modality gap in visual planning improves VLM performance by leveraging symbolic data during training.
• STRIDE: Strategic Trajectory Reasoning via Discriminative Estimation for Verifiable Reinforcement Learning — STRIDE enhances RLVR by using verifiable outcomes to identify and reward outcome-discriminative strategic patterns, improving credit assignment.
• WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving — WCog-VLA enables proactive autonomous driving by integrating semantic world forecasting with generative multi-agent trajectory evolution.
• Probing the Reliability of Driving VLMs: From Inconsistent Responses to Grounded Temporal Reasoning — Driving VLMs exhibit response inconsistency and limited temporal reasoning, hindering their reliability as assistants.
• VeriDrive: Verifiable Counterfactual Supervision for Cost-Efficient Vision-Language Planning — Structured, verifiable counterfactual supervision significantly improves vision-language planning efficiency and performance in autonomous driving.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.