Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making

2026-04-15 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

This survey explores the integration of Explainable Artificial Intelligence (XAI) with surrogate modeling for complex system simulations, addressing the inherent opacity of black-box simulators. Surrogate models, while reducing computational costs, often exacerbate this black-box nature, making it difficult to understand how inputs influence physical responses. The paper maps existing XAI techniques to different stages of surrogate modeling workflows, drawing on examples from equation-based and agent-based simulations. It highlights XAI's strengths in revealing interactions and enhancing human comprehension, while also identifying challenges such as explaining dynamical and mixed-variable systems. The authors propose a research agenda to embed explainability throughout simulation-driven workflows, aiming to transform opaque emulators into tools for extracting actionable insights.

Key takeaway

For AI Scientists and Machine Learning Engineers working with complex simulations, integrating XAI into your surrogate modeling workflows is crucial. This approach moves beyond mere computational acceleration, enabling you to extract actionable insights and understand the underlying mechanisms of your systems. Prioritize XAI methods that address challenges like highly correlated inputs and dynamical systems to enhance model transparency and decision-making.

Key insights

Integrating XAI with surrogate models enhances transparency and insight extraction from complex simulations.

Principles

• Surrogate models inherit black-box opacity.
• XAI can unpack complex model behaviors.
• Explainability should be embedded in workflows.

Method

The survey maps XAI techniques to surrogate modeling workflow stages, synthesizing applications across equation-based and agent-based simulations to reveal interactions and support comprehension.

In practice

• Apply XAI to understand input-response links.
• Use XAI for dynamical system analysis.
• Integrate XAI in model construction.

Topics

• Surrogate Modeling
• Explainable AI
• Complex Systems Simulation
• Dynamical Systems
• Agent-Based Modeling

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making — Integrating XAI with surrogate models enhances transparency and diagnostic capabilities for complex simulations, bridging efficiency and interpretability.
• Radically Better Reasoning: Elicit's Andreas Stuhlmüller & Jungwon Byun on World Models for Research — Elicit ensures trusted AI reasoning in scientific research via process supervision, domain-specific primitives, and inspectable world models.
• IEEE Transactions on Evolutionary Computation, Volume 30, Issue 1, February 2025 — Evolutionary computation continues to advance, integrating with AI and deep learning for complex optimization and scheduling tasks.
• Concerning Uncertainty -- A Systematic Survey of Uncertainty-Aware XAI — UAXAI integrates uncertainty into AI explanations to improve transparency, reliability, and human trust in high-stakes decision-making.
• Explainable AI for Data-Driven Design of High-Dimensional Predictive Studies — Explainable AI can optimize high-dimensional predictive study design by recommending feature engineering for interpretable models.
• Towards the explainability of protein language models — XAI is crucial for understanding and advancing protein language models, which currently operate as black boxes.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.