A Survey of Reasoning in Autonomous Driving Systems: Open Challenges and Emerging Paradigms
Summary
A survey by researchers from Tsinghua University, East China Normal University, and The University of Hong Kong argues that the primary bottleneck for high-level autonomous driving (AD) has shifted from perception to robust, generalizable reasoning. Current AD systems struggle with complex, long-tail scenarios and social interactions requiring human-like judgment. The paper proposes a novel Cognitive Hierarchy to decompose driving tasks into Sensorimotor, Egocentric Reasoning, and Social-Cognitive levels, and identifies seven core reasoning challenges, including heterogeneous signal reasoning, perception-cognition bias, and the responsiveness-reasoning trade-off. It reviews state-of-the-art system-centric approaches, highlighting a trend towards holistic, interpretable "glass-box" agents, and evaluates current benchmarks, noting a shift from physical outcomes to cognitive process assessment. The authors conclude that integrating high-latency, deliberative LLM-based reasoning with millisecond-scale vehicle control remains a fundamental, unresolved tension.
Key takeaway
For AI Scientists and Research Scientists developing autonomous driving systems, you should prioritize integrating advanced reasoning capabilities, particularly for social-cognitive challenges. Focus on developing verifiable neuro-symbolic architectures that can reconcile the latency of large language models with real-time vehicle control demands, and design evaluation frameworks that rigorously test generalization in long-tail and safety-critical scenarios. Your efforts must bridge the gap between abstract reasoning and physically grounded, socially compliant actions to achieve higher levels of autonomy.
Key insights
Autonomous driving's core challenge is shifting from perception to robust, human-like reasoning, especially in complex social scenarios.
Principles
- Reasoning must be the cognitive core, not a modular component.
- Dual-process architectures balance fast reactions with deep deliberation.
- Prioritize compliance and social acceptability over pure performance.
Method
The proposed Cognitive Hierarchy deconstructs driving into Sensorimotor, Egocentric, and Social-Cognitive levels, systematizing seven core reasoning challenges for LLM integration.
In practice
- Develop verifiable neuro-symbolic architectures for safety.
- Design systems to dynamically query external regulatory knowledge.
- Use generative and adversarial evaluation to find novel failure modes.
Topics
- Autonomous Driving Reasoning
- Large Multimodal Models
- Cognitive Architectures
- Long-Tail Scenarios
- Neuro-Symbolic AI
Best for: AI Scientist, Research Scientist, AI Researcher, AI Engineer, Robotics Engineer
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.