Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning

2026-06-08 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, extended

Summary

A new metric, Normalized Logit Difference Decay (NLDD), quantifies the causal influence of Chain-of-Thought (CoT) reasoning steps on Large Language Model predictions. This approach corrupts individual reasoning steps and measures the model's confidence drop, standardizing results for cross-model comparison. Evaluating DeepSeek-Coder-6.7B-Instruct, Llama-3.1-8B-Instruct, and Gemma-2-9B-Instruct across syntactic (Dyck-$n$), logical (PrOntoQA), and arithmetic (GSM8K) tasks, researchers identified a consistent "Reasoning Horizon ($k^{*}$)" at 70–85% of the chain length, beyond which reasoning tokens have negligible or negative impact. The study revealed "Faithful" regimes (Llama, DeepSeek) where CoT is causal, and an "Anti-Faithful" regime (Gemma), where Gemma achieved 99.0% accuracy on PrOntoQA with a negative NLDD of -52.5%, indicating corruption paradoxically increased confidence. A "Mapping Gap" was also observed, where models internally encode task-relevant information (e.g., 82.0% probe accuracy for Gemma on Dyck-$n$) but fail to utilize it for the final answer (0.0% task accuracy).

Key takeaway

For Machine Learning Engineers deploying LLMs with Chain-of-Thought, you should integrate NLDD to assess true reasoning faithfulness, not just accuracy. Your models might be "anti-faithful," achieving high performance without causally depending on their CoT, or exhibiting a "Mapping Gap." Use the identified Reasoning Horizon ($k^{*}$) to prune unnecessary reasoning steps, potentially improving efficiency or preventing negative interference from extended CoT, especially in critical applications.

Key insights

NLDD quantifies CoT faithfulness, revealing models can achieve high accuracy without causally relying on their generated reasoning.

Principles

• CoT faithfulness varies significantly across LLM architectures.
• High accuracy does not guarantee causal reliance on CoT.
• Reasoning influence decays beyond a specific "horizon."

Method

NLDD measures confidence degradation in logit space when individual CoT steps are corrupted, normalizing for cross-model comparison. It complements this with Representational Similarity Analysis (RSA) and Trajectory Alignment Score (TAS) to probe internal states.

In practice

• Use NLDD to identify causally irrelevant CoT steps.
• Prune CoT chains beyond the detected Reasoning Horizon ($k^{*}$).
• Evaluate models for "anti-faithful" behavior, not just accuracy.

Topics

• Chain-of-Thought
• LLM Faithfulness
• NLDD Metric
• Reasoning Horizon
• Model Architectures
• Interpretability Methods

Code references

• donald-ye/ACL_2026

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structures — LLMs' apparent faithfulness to intermediate reasoning structures is fragile; they often fail to update predictions after causal intervention.
• Feedback request: Testing the $H_{dp}$ bandwidth bound on LLM benchmarks (Preprint check & review) — The $H_{dp}$ bandwidth bound predicts CoT benefits only when task depth exceeds a transformer's single-pass capacity.
• Reliable Chain-of-Thought via Prefix Consistency — Truncating and regenerating LLM Chain-of-Thought traces reveals a reliability signal for improved answer aggregation.
• Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations — LLM robustness to CoT perturbations varies significantly by type and model scale, challenging assumptions that size alone ensures reliability.
• Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models — Large language models exhibit a "commitment boundary" in Chain-of-Thought reasoning, after which subsequent steps are epiphenomenal.
• LLM Reasoning Is Latent, Not the Chain of Thought — LLM reasoning is best understood as latent-state trajectory formation, not explicit chain-of-thought.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.