Explainable embeddings with Distance Explainer

2026-07-08 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

The paper introduces Distance Explainer, a novel local, post-hoc eXplainable AI (XAI) method for interpreting embedded vector spaces. Adapting saliency-based techniques from RISE, it explains the distance between two embedded data points by assigning attribution values through selective masking and distance-ranked mask filtering. Evaluated on cross-modal embeddings (image-image and image-caption pairs) using ImageNet (1000-dimensional classification vectors) and CLIP (512-dimensional semantic vectors) models, the method effectively identifies features contributing to similarity or dissimilarity. Experiments with ResNet50 and VGG16 models demonstrate high robustness and consistency, assessed via Faithfulness, Sensitivity/Robustness, and Randomization metrics. Parameter tuning, including mask quantity and selection strategy, significantly affects explanation quality.

Key takeaway

For Machine Learning Engineers developing or deploying models with embedded spaces, understanding feature contributions to embedding distances is critical for trustworthiness. You should consider integrating Distance Explainer to generate local, post-hoc explanations for similarity or dissimilarity between data points. Experiment with mask quantity and selection strategies to optimize explanation quality for your specific cross-modal or single-modal applications, enhancing model transparency.

Key insights

Distance Explainer attributes feature contributions to similarity or dissimilarity between embedded data points.

Principles

• XAI for embedded spaces requires local, post-hoc explanations.
• Distance-ranked mask filtering improves saliency map generation.
• Parameter tuning is crucial for explanation quality.

Method

Distance Explainer masks an input item, calculates its cosine distance to a reference item's embedding, ranks masks by distance, filters them (top/bottom X%), and sums remaining masks to create an attribution map.

In practice

• Explain image-image similarity using ImageNet models.
• Interpret image-caption semantic relationships with CLIP.
• Tune mask quantity and selection for specific use cases.

Topics

• Explainable AI
• Embedded Spaces
• Saliency Maps
• Multi-modal Embeddings
• CLIP Model
• Attribution Methods

Code references

• dianna-ai/distance_explainer
• dianna-ai/explainable_embedding

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability — Unimodal VLM explainability metrics fail due to cross-modal redundancy; synergistic metrics are crucial for faithful evaluation.
• Alike Parts: A Feature-Informed Approach to Local and Global Prototype Explanations — Integrating feature importance into prototype selection enhances both local explanations and global prototype diversity without compromising model fidelity.
• Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures — Intrinsic interpretability builds transparency directly into LLM architectures to enhance trustworthiness and safety.
• SketchXplain: Intuitive Visual Explanations of Image Classifiers with Sketches — SketchXplain generates intuitive, sketch-based visual explanations for image classifiers by combining saliency, concept models, and sketch optimization.
• OPTIMUS-Prime: Minimal and Sufficient Concept Explanations for Deep Vision Models — OPTIMUS-Prime provides minimal and sufficient concept-based visual explanations for deep vision models, grounded in prime implicant theory.
• Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making — Integrating XAI with surrogate models enhances transparency and diagnostic capabilities for complex simulations, bridging efficiency and interpretability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.