From Plausible to Actionable: A Position on LLM Self-Explanations

2026-07-17 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Advanced, quick

Summary

Large Language Models (LLMs) can generate natural language self-explanations to rationalize their decisions, a promising direction for explainable artificial intelligence (XAI). While these explanations often appear plausible, their faithfulness to the model's true underlying reasoning remains uncertain. This analysis argues that self-explanations can be highly plausible, questionably faithful, yet profoundly actionable. It identifies limitations in standard evaluation protocols for LLM self-explanations. The paper also proposes practical guidelines for assessing plausibility and faithfulness. Furthermore, it advocates extending evaluation beyond these criteria to include actionability. This highlights how LLM rationalizations support informed decision-making for diverse stakeholders.

Key takeaway

For AI scientists and practitioners evaluating LLM outputs, recognize that LLM self-explanations, even if their faithfulness to internal reasoning is questionable, offer significant value for informed decision-making. You should prioritize evaluating these explanations based on their actionability and practical utility, rather than solely on their perceived faithfulness. This shift ensures that LLM rationalizations effectively support real-world applications and stakeholder needs.

Key insights

LLM self-explanations are valuable for decision-making and action, even with questionable faithfulness.

Principles

• Self-explanations can be plausible but not necessarily faithful.
• Evaluation must extend beyond plausibility and faithfulness to actionability.

Method

Propose practical guidelines for assessing the plausibility and faithfulness of LLM-generated self-explanations.

In practice

• Support informed decision-making across stakeholders.
• Guide appropriate action based on LLM rationalizations.

Topics

• Large Language Models
• LLM Self-Explanations
• Explainable AI
• Model Evaluation
• Actionability
• AI Rationalization

Best for: Research Scientist, AI Product Manager, AI Scientist, AI Ethicist, Director of AI/ML

Related on AIssential

• From Plausible to Actionable: A Position on LLM Self-Explanations — LLM self-explanations are plausible but unfaithful; their true value lies in their actionability for informed decision-making.
• A Systematic Comparison between Extractive Self-Explanations and Human Rationales in Text Classification — LLM self-explanations offer faithful rationales, differing from post-hoc methods, but their human alignment varies with task complexity.
• A Definition of Good Explanations and the Challenges Explaining LLM Outputs — Good AI explanations combine counterfactuals with the interlocutor's prior beliefs to address LLM explainability challenges.
• LLMs Can’t Provide Faithful Explanations Needed for AI Accountability — LLMs struggle to provide faithful explanations, hindering accountability in high-stakes AI applications.
• Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures — Intrinsic interpretability builds transparency directly into LLM architectures to enhance trustworthiness and safety.
• Project Ariadne: A Structural Causal Framework for Auditing Faithfulness in LLM Agents — LLM agent reasoning traces often act as "Reasoning Theater," not faithful generative drivers, due to Causal Decoupling.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.