From Plausible to Actionable: A Position on LLM Self-Explanations

2026-07-20 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, extended

Summary

A position paper argues that Large Language Model (LLM) self-explanations, while often appearing plausible, are questionably faithful to the model's true reasoning process. Current XAI evaluation protocols for self-explanations have significant limitations, including their failure to consider stakeholder expertise or embrace explanation variation for plausibility, and their unsuitability for LLMs due to nondeterminism, prompt sensitivity, and reliance on internal knowledge for faithfulness. The paper advocates shifting the primary focus from plausibility and faithfulness to "actionability", defining it as the capacity of self-explanations to support informed decision-making and enable appropriate actions across diverse stakeholders. This actionable value is realized by positioning self-explanations as communicative interfaces for non-experts, tools for supporting human judgment, and facilitators of deliberation through diverse perspectives.

Key takeaway

For AI scientists and Machine Learning Engineers developing LLM-based systems for high-stakes applications, you should prioritize the "actionability" of self-explanations over their perceived faithfulness. Focus on designing explanations that serve as critical arguments for human decision-makers, communicate uncertainty, and facilitate diverse perspectives, rather than striving for unachievable faithful introspection. This approach mitigates automation bias and fosters informed trust, ensuring LLMs support, not replace, human judgment.

Key insights

LLM self-explanations are plausible but unfaithful; their true value lies in their actionability for informed decision-making.

Principles

• Plausibility evaluation must consider stakeholder expertise and embrace explanation variation.
• Faithfulness evaluation for LLMs must account for nondeterminism and prompt sensitivity.
• Self-explanations should be viewed as arguments, not literal accounts of model reasoning.

Method

The paper proposes practical guidelines for assessing plausibility and faithfulness, then advocates a research agenda repositioning self-explanations as actionable communicative interfaces, decision-support tools, and deliberation facilitators.

In practice

• Use self-explanations to translate complex XAI into natural language for non-experts.
• Design LLMs with safety protocols to generate arguments, not final decisions.
• Employ multiple LLMs as "advocates" to surface diverse viewpoints for deliberation.

Topics

• Large Language Models
• Explainable AI
• LLM Self-Explanations
• XAI Evaluation
• Actionability
• Human-AI Decision Support

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Ethicist

Related on AIssential

• From Plausible to Actionable: A Position on LLM Self-Explanations — LLM self-explanations are valuable for decision-making and action, even with questionable faithfulness.
• A Definition of Good Explanations and the Challenges Explaining LLM Outputs — Good AI explanations combine counterfactuals with the interlocutor's prior beliefs to address LLM explainability challenges.
• LLMs Can’t Provide Faithful Explanations Needed for AI Accountability — LLMs struggle to provide faithful explanations, hindering accountability in high-stakes AI applications.
• A Systematic Comparison between Extractive Self-Explanations and Human Rationales in Text Classification — LLM self-explanations offer faithful rationales, differing from post-hoc methods, but their human alignment varies with task complexity.
• People need to understand why systems make decisions, not merely whether the systems appear to perform well. A map of the Explainable AI (xAI) landscape. — Explainability is now a prerequisite for trustworthy AI, demanding tailored methods for diverse contexts.
• Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures — Intrinsic interpretability builds transparency directly into LLM architectures to enhance trustworthiness and safety.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.