Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework

2026-07-11 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

GraphEVAL, a novel graph-based reasoning framework, quantifies uncertainty, coherence, and robustness in Large Language Model (LLM) reasoning, addressing the limitations of traditional decoding strategies like Self-Consistency (SC). It introduces the Graph Reasoning Coherence Score (GRCS), a distributional uncertainty metric based on Structural-Semantic Graph Edit Distance (SS-GED), which consistently shows a negative correlation with reasoning faithfulness across models like LLaMA 3.1 8B, Phi-4, and DeepSeek R1. GraphEVAL also proposes Graph Self-Consistency (GSC), a medoid-based decoding strategy that improves reasoning fidelity, exposing "lucky guesses" in smaller models while preserving or enhancing accuracy in more capable ones. The framework's SS-GED computation has a time complexity of O(m^3), a significant improvement over O(m^m) for exact GED. Experiments across diverse tasks, including StrategyQA, MedQA, and GPQA, demonstrate that the GSC-selected path acts as a "load-bearing path," where adversarial ablation degrades reasoning faithfulness and, in targeted cases, accuracy.

Key takeaway

For Machine Learning Engineers deploying LLMs in mission-critical environments, relying solely on final answer agreement for trustworthiness is insufficient and risky. You should integrate reasoning-level analysis using frameworks like GraphEVAL to quantify uncertainty and ensure logical fidelity. This approach helps identify "lucky guesses" and unfaithful reasoning, providing a more robust evaluation than traditional Self-Consistency, especially for smaller models where mode collapse and hallucinations are prevalent.

Key insights

LLM trustworthiness hinges on evaluating reasoning fidelity, not just final answer agreement, a gap GraphEVAL addresses.

Principles

• Quantifying reasoning coherence and uncertainty is essential for true LLM trustworthiness.
• Majority voting on final outputs (Self-Consistency) is vulnerable to unfaithful reasoning and "lucky guesses".
• Modeling Chain-of-Thoughts as DAGs reveals critical signals of uncertainty, faithfulness, and robustness.

Method

GraphEVAL models sampled Chain-of-Thoughts as DAGs, decomposes steps into atomic facts, and quantifies semantic-structural consensus using Graph Reasoning Coherence Score (GRCS) based on Structural-Semantic Graph Edit Distance (SS-GED) to select the most coherent path.

In practice

• Apply GRCS to reliably quantify uncertainty and detect confident hallucinations in LLM reasoning.
• Implement Graph Self-Consistency (GSC) decoding to filter out unfaithful "lucky guesses" and improve reasoning fidelity.
• Use adversarial medoid ablation to probe the robustness of LLM reasoning paths.

Topics

• Large Language Models
• Uncertainty Quantification
• Reasoning Fidelity
• Graph-Based AI
• Chain-of-Thought
• LLM Hallucinations

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework — GRAPHEVAL quantifies LLM reasoning uncertainty and improves fidelity by analyzing intermediate steps via graph-based coherence and a medoid-based decoding strategy.
• Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations — LLM robustness to CoT perturbations varies significantly by type and model scale, challenging assumptions that size alone ensures reliability.
• The Computational Basis of Confidence in Large Language Models — Answer-logit differences in LLMs can reflect a latent decision variable, aligning with statistical decision confidence.
• Trust Me. I’m Artificial Intelligence. — AI models are becoming more convincing in their reasoning, but this does not equate to increased truthfulness or correctness.
• Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning — NLDD quantifies CoT faithfulness, revealing models can achieve high accuracy without causally relying on their generated reasoning.
• From Wordle to Fibble5: Evaluating LLM Reasoning Under Escalating Deception — LLM reasoning robustness is severely degraded by even minimal deception, a factor poorly captured by standard benchmarks.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.