LLM-Based Visual Explanation Evaluation Framework for Assessing the Explainability of Facial Skin Disease Classification Models

2026-06-15 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Expert, quick

Summary

A new domain-specific LLM-based Visual Explanation Evaluation Framework is proposed to assess visual attention explanations in facial skin disease diagnosis. This framework addresses the gap where prior research focused on classification performance rather than the clinical relevance of visual explanations. It employs an image-driven algorithm that generates lesion-focused attention maps by combining color saliency, facial spatial priors, Gaussian smoothing, and attention overlay visualization, yielding clinically interpretable results. Furthermore, an LLM-as-a-Judge evaluation component utilizes GPT-5.5, Gemini 3.5 Flash, and Claude Sonnet 4.6 to evaluate these visual explanations for lesion localization and trustworthiness.

Key takeaway

For AI Scientists developing explainable AI for medical imaging, this framework offers a robust method to ensure visual explanations are clinically relevant. You should consider integrating domain-specific priors and LLM-as-a-Judge components, utilizing models like GPT-5.5 or Gemini 3.5 Flash, to validate explanation trustworthiness and accurate lesion localization, moving beyond mere classification performance metrics.

Key insights

A framework uses LLMs to evaluate clinically relevant visual explanations for facial skin disease diagnosis.

Principles

• Visual explanations require clinical grounding for trustworthiness
• Combine color saliency with facial spatial priors
• LLMs can serve as effective evaluators for visual explanations

Method

Develop an image-driven algorithm using color saliency, facial spatial priors, Gaussian smoothing, and attention overlay. Evaluate explanations with an LLM-as-a-Judge framework using GPT-5.5, Gemini 3.5 Flash, and Claude Sonnet 4.6.

In practice

• Generate lesion-focused attention maps for medical images
• Assess explanation trustworthiness using LLM-based evaluation

Topics

• Explainable AI
• Facial Skin Disease
• Visual Explanations
• LLM-as-a-Judge
• Medical Imaging
• Attention Maps
• GPT-5.5

Best for: Computer Vision Engineer, AI Scientist, Research Scientist

Related on AIssential

• LLM-Based Visual Explanation Evaluation Framework for Assessing the Explainability of Facial Skin Disease Classification Models — An LLM-based framework evaluates Grad-CAM explanations for facial skin disease models, ensuring clinical relevance and trustworthiness.
• GLARE: A Natural Language Interface for Querying Global Explanations — LLM-based interfaces can transform complex global explanations into accessible, interactive natural language queries for black-box models.
• Beyond Heatmaps: Unsupervised Concept-Graph Reasoning for Interpretable Visual Explanation — G-CBM offers intrinsically interpretable visual explanations by discovering concepts unsupervisedly and modeling their dependencies via graph reasoning.
• Explainable Iterative Data Visualisation Refinement via an LLM Agent — An LLM-driven agent automates and explains data visualization refinement by iteratively optimizing DR hyperparameters.
• The mirage of visual understanding in current frontier models — Current LLMs can generate convincing visual descriptions and reasoning without actual image input, termed "mirage reasoning."
• Local Label-Informed Feature Transfer for Generating Ground-Truth Medical Images: A Comparison of GAN- and Diffusion-Based Approaches — The LLIFT framework generates realistic, semi-synthetic medical images with controlled lesions, bypassing pixel-level annotation needs.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.