Making AI systems more transparent and trustworthy: an interview with Ximing Wen

2026-05-07 · Source: ΑΙhub · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

Ximing Wen, a PhD candidate at Drexel University, is researching methods to enhance the transparency and trustworthiness of AI systems, particularly large language models. Her work addresses the challenge of models providing confident but unverified answers by developing approaches that show reasoning and evidence for outputs, crucial for applications in healthcare and legal document review. Wen's research includes a prototype-based approach that achieves interpretability without sacrificing accuracy, outperforming previous interpretable models. She extended this concept to generative models and medical AI, creating interpretable diagnostic tools for limited datasets. A key finding involved redesigning a loss function for spatial coordinates, boosting accuracy from 65% to over 85% in document understanding tasks, emphasizing the importance of teaching methods in model performance.

Key takeaway

For AI Scientists and Research Scientists developing language models for sensitive domains like healthcare or legal review, you should prioritize integrating transparency mechanisms from the outset. Your models must not only provide accurate answers but also clearly show their reasoning and supporting evidence. Consider adopting prototype-based interpretability and spatial grounding techniques to build trust and enable verification, especially when working with limited training data or aiming to explain reward model preferences for safer AI systems.

Key insights

Interpretable AI models can achieve high accuracy by showing reasoning and evidence, crucial for trust.

Principles

• Interpretability need not sacrifice accuracy.
• Teaching methods are as vital as the model itself.

Method

Develop prototype-based approaches for classification and spatial grounding for document Q&A to show model reasoning and evidence, even with limited data.

In practice

• Use prototype-based reasoning for classification.
• Apply spatial grounding for document question answering.
• Integrate prototype reasoning into reward models.

Topics

• Transparent AI
• Trustworthy AI
• Language Models
• Prototype-based Interpretability
• Spatial Grounding

Best for: AI Scientist, Research Scientist, AI Ethicist

Related on AIssential

• Why Should I Trust You? (and Can I?) — AI models are black boxes because their complex internal reasoning is difficult to interpret, not because their underlying mechanics are unknown.
• Show Your Work: The Case for Radical AI Transparency — Radical AI transparency, sharing full AI interaction threads, builds trust and clarifies human judgment.
• Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures — Intrinsic interpretability builds transparency directly into LLM architectures to enhance trustworthiness and safety.
• New RFP on Interpretability from Schmidt Sciences — Schmidt Sciences has launched a "Request for Proposals" (RFP) for a pilot program in AI interpretability, seeking methods to detect and mitigate deceptive behaviors in "Large Language Models" (LLMs) by May 26, 2026.
• Prototype-Grounded Concept Models for Verifiable Concept Alignment — PGCMs enhance CBM interpretability by grounding abstract concepts in verifiable visual prototypes, enabling direct inspection and intervention.
• Towards Faithful Agentic XAI: A Verification Method and an Open-World Benchmark for Better Model Faithfulness — Agentic XAI requires explicit verification against model behavior to ensure explanation faithfulness and prevent user misinformation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ΑΙhub.