Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making

2026-04-17 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

This survey paper, "Interpretable and Explainable Surrogate Modeling for Simulations," addresses the critical gap between computationally efficient but opaque black-box simulators and the need for transparent, understandable models in scientific and engineering domains. It systematically integrates Explainable Artificial Intelligence (XAI) techniques into surrogate modeling workflows, which are typically used to reduce the computational cost of complex simulations like equation-based and agent-based models. The paper maps existing XAI methods to various stages of surrogate modeling for design and exploration, drawing on applications in fields such as urban systems and aerospace. It highlights the strengths of XAI in revealing interactions and supporting human comprehension, while also identifying open challenges like explaining dynamical and mixed-variable systems. The authors propose a research agenda to embed explainability as a core element throughout simulation-driven workflows, from model construction to decision-making, aiming to transform opaque emulators into actionable, transparent tools.

Key takeaway

For Machine Learning Engineers and Research Scientists developing or deploying simulation-driven models, you should prioritize integrating XAI methods directly into your surrogate modeling workflows. This approach will not only enhance model trustworthiness and validation but also enable the extraction of actionable insights from complex system behaviors, moving beyond mere computational efficiency to truly informed decision-making. Focus on combining global and local explainability techniques to gain a comprehensive understanding of model dynamics and ensure your explanations are robust and uncertainty-aware.

Key insights

Integrating XAI with surrogate models enhances transparency and diagnostic capabilities for complex simulations, bridging efficiency and interpretability.

Principles

• Explainability must be embedded, not post-hoc, in simulation workflows.
• Balance global sensitivity with local feature attribution for comprehensive understanding.
• No single XAI method is universally optimal; context-aware strategies are crucial.

Method

The proposed workflow integrates XAI across data acquisition, global exploration, input-output analysis, dimensionality reduction, and robust design, using techniques like GSA, PDP, ICE, and SHAP.

In practice

• Use Latin Hypercube Sampling for efficient, uniform exploration of parameter spaces.
• Apply SHAP values to decompose multi-objective trade-offs quantitatively.
• Leverage interactive dashboards and VR/AR for immersive XAI communication.

Topics

• Explainable AI
• Surrogate Modeling
• Complex Systems Simulations
• Global Sensitivity Analysis
• Shapley Values

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making — Integrating XAI with surrogate models enhances transparency and insight extraction from complex simulations.
• People need to understand why systems make decisions, not merely whether the systems appear to perform well. A map of the Explainable AI (xAI) landscape. — Explainability is now a prerequisite for trustworthy AI, demanding tailored methods for diverse contexts.
• Towards the explainability of protein language models — XAI is crucial for understanding and advancing protein language models, which currently operate as black boxes.
• Explaining AI Without Code: A User Study on Explainable AI — Integrating XAI into no-code ML platforms improves usability and trust, but must balance novice accessibility with expert diagnostic depth.
• Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models — CA-LIG provides context-aware, layer-wise explanations for Transformer models by integrating token attributions with attention gradients.
• SAILS: Surrogate-based Analysis of Interactions via Local Effect Smooths — SAILS characterizes pairwise feature interactions in black-box models by fitting GAM surrogates to local effects, enabling detection, categorization, and visualization of functional forms.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.