ConceptSMILE: Auditing the Trustworthiness of Concept-Based Explainable AI

2026-07-10 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Expert, quick

Summary

ConceptSMILE is a new model-agnostic, perturbation-based auditing framework designed to evaluate the trustworthiness of concept-based explainable AI (XAI). This framework extends the logic of SMILE from feature- or region-level attribution to human-understandable concept explanations. ConceptSMILE operates by perturbing input regions, measuring shifts in concept responses, applying locality weighting, and fitting an XGBoost surrogate to approximate local concept behavior. It assesses reliability across five metrics: attribution accuracy, surrogate fidelity, faithfulness, stability, and consistency. An evaluation on retinal fundus images compared MedSAM-derived visual concepts with VLM-based semantic concepts. Results indicate varying reliability, with MedSAM achieving stronger spatial attribution and the highest surrogate fidelity ($R^2 = 0.8503$, $R_w^2 = 0.8465$). The VLM pathway demonstrated stronger vessel faithfulness and stability under specific artifact conditions, establishing ConceptSMILE as an independent audit layer for concept-based XAI.

Key takeaway

For AI Scientists and Machine Learning Engineers developing or deploying concept-based XAI, you should integrate ConceptSMILE as an independent auditing layer to rigorously evaluate explanation trustworthiness. This framework helps you identify variations in reliability across different concepts and pathways, such as those from MedSAM or VLM. By assessing attribution accuracy, fidelity, faithfulness, stability, and consistency, you can ensure your concept-based explanations are robust and dependable, especially in critical applications like medical imaging.

Key insights

ConceptSMILE provides a model-agnostic framework to audit the trustworthiness of concept-based XAI by perturbing inputs and measuring concept response shifts.

Principles

• Concept-level outputs require independent auditing.
• Reliability varies across concepts and pathways.
• Perturbation-based logic extends to concepts.

Method

Perturb input regions, measure concept-response shifts, apply locality weighting, and fit an XGBoost surrogate to approximate local concept behavior for reliability assessment.

In practice

• Compare MedSAM and VLM concept pathways.
• Evaluate XAI reliability using five metrics.
• Audit concept-based explanations for trustworthiness.

Topics

• Concept-based XAI
• Explainable AI Auditing
• Model Agnostic XAI
• Retinal Fundus Images
• MedSAM
• XGBoost Surrogate Models

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models — The framework unifies XAI evaluation by scoring fidelity, simplicity, and stability to build a knowledge base for transparent, trustworthy AI.
• When Confidence Lacks Concepts: Interpretable OOD Detection via Representation Perturbations — Interpretable OOD detection can be achieved by analyzing prediction stability under concept-conditioned semantic perturbations.
• From Sparse Features to Trustworthy Proxies: Certifying SAE-Based Interpretability — A framework certifies SAE-based LM interpretability by bounding risk using proxy fidelity, reconstruction, mismatch, and complexity.
• Auditing Construct Overlap in Explainable Machine Learning: Evidence from Burnout-Depression Prediction Across Student Cohorts — XAI pipelines can produce misleadingly stable risk hierarchies due to construct overlap in mental health outcomes.
• Interview with Deepika Vemuri: interpretability and concept-based learning — Concept-based learning enhances AI interpretability by aligning model representations with human-understandable concepts.
• Making AI systems more transparent and trustworthy: an interview with Ximing Wen — Interpretable AI models can achieve high accuracy by showing reasoning and evidence, crucial for trust.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.