Output Type Before Quality: A Standards-Derived XAI Admissibility Rubric for Autonomous-Driving Safety

2026-06-06 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

The article introduces the "evidence-type gap" in autonomous driving (ADS) safety assurance, where XAI methods' output types (e.g., ranked feature lists from SHAP) do not match the causal evidence required by safety standards (e.g., directed cause-and-effect chains from ISO/PAS 8800 Cl. 6.7.1). The authors derive 19 testable evidentiary criteria across seven lifecycle stages from four publications: AMLAS, ISO 21448, ISO/PAS 8800, and ISO 26262. They structurally score six XAI method classes (SHAP, GradCAM, Counterfactual, SCM, Causal Trace, CoC Trace) based on whether their output type can satisfy these criteria. Causal XAI, specifically Structural Causal Models (SCM), is found to be structurally required for hazard identification (+62% rubric gap), incident investigation (+50%), and data management (+50%), with verdicts stable across thresholds T\u2208(0%,50%] and robust to single-cell flips down to T=25%. A proof-of-concept on 1,996 real-world driving clips (79,840 rows) confirms that observed output types match rubric predictions.

Key takeaway

For AI Architects or Machine Learning Engineers developing autonomous driving systems, selecting XAI methods based on popularity risks non-compliance with safety standards. You should evaluate XAI methods against specific lifecycle-stage evidentiary demands, prioritizing causal XAI, like Structural Causal Models, for critical stages such as hazard identification, incident investigation, and data management. This ensures your chosen methods can structurally provide the required causal evidence, rather than merely associational outputs.

Key insights

Autonomous driving safety standards demand causal evidence that many popular XAI methods cannot structurally provide.

Principles

• XAI method selection must align with lifecycle-stage evidence demand.
• Pearl's causal hierarchy dictates which questions XAI methods can answer.
• Output type, not just quality, determines XAI method admissibility.

Method

A rubric of 19 criteria, derived from four safety standards, scores XAI methods structurally (S/P/F) based on their output type's ability to meet evidentiary demands across seven lifecycle stages.

In practice

• Prioritize Causal XAI for hazard identification and incident investigation.
• Use Structural Causal Models (SCM) for stages requiring directed causal paths.
• Evaluate XAI methods against specific safety standard clauses.

Topics

• Autonomous Driving Safety
• Explainable AI
• Causal Inference
• ISO 26262
• Structural Causal Models
• Evidence-Type Gap

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Scientist, Machine Learning Engineer, AI Architect

Related on AIssential

• Output Type Before Quality: A Standards-Derived XAI Admissibility Rubric for Autonomous-Driving Safety — Autonomous driving safety standards demand specific evidence types that popular XAI methods often cannot provide, necessitating a shift to causal XAI for critical stages.
• Causality-Aware End-to-End Autonomous Driving via Ego-Centric Joint Scene Modeling — Causality-aware modeling improves end-to-end autonomous driving by capturing ego-agent interdependencies for consistent trajectory prediction.
• Formal verification for safety evaluation of autonomous vehicles: an interview with Abdelrahman Sayed Sayed — Formal verification of neural ODEs can provide critical safety guarantees for AI in autonomous systems.
• On the Road With DRIVE AV — A hybrid AV stack combining end-to-end AI with a classical safety system offers both human-like driving and robust safety.
• A Survey of Reasoning in Autonomous Driving Systems: Open Challenges and Emerging Paradigms — Autonomous driving's core challenge is shifting from perception to robust, human-like reasoning, especially in complex social scenarios.
• The Blind Spot in AI Safety — AI safety research's focus on coherent vs. incoherent failure overlooks "epistemic drift," a critical unmeasured failure mode.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.