SpanUQ: Span-Level Uncertainty Quantification for Large Language Model Generation

2026-07-08 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

SpanUQ is a lightweight, ~25M-parameter probe designed for Span-Level Uncertainty Estimation (SLUE) in large language model (LLM) generation, addressing the limitations of existing token- and sequence-level methods. It distills uncertainty knowledge from expensive multi-sample inference into a single forward pass over LLM hidden states. Utilizing a DETR-style span decoder and a Mixture of Beta distribution, SpanUQ simultaneously detects semantically coherent text spans and estimates their continuous uncertainty. Evaluated on SpanUQ-Bench, the first span-level uncertainty benchmark with ~293K annotated spans, SpanUQ consistently achieved AUROC 0.908–0.944 and MAE 0.110–0.129 across five LLM backbones (Qwen3-14B/8B/4B/30B-A3B, Mistral-7B). It is 10–20x faster than sampling-based methods, with its DETR detector reaching 0.910 F1, a 39.4% improvement over heuristics.

Key takeaway

For AI Scientists and ML Engineers deploying LLMs in high-stakes applications, SpanUQ offers a critical advancement for ensuring trustworthiness. You should consider integrating this lightweight probe to gain fine-grained, span-level uncertainty estimates, which are 10–20x faster than sampling methods. This enables precise localization of potentially erroneous claims, facilitating targeted human review or automated self-refinement. However, remember that SpanUQ estimates epistemic uncertainty from internal signals; it complements, but does not replace, external verification for factual correctness.

Key insights

SpanUQ provides fine-grained, efficient uncertainty estimates for LLM outputs by detecting semantically coherent text spans.

Principles

• Uncertainty estimation is most effective at semantically coherent span granularity.
• Multi-layer fusion of LLM hidden states captures diverse uncertainty signals.
• Mixture of Beta distributions accurately models multimodal uncertainty.

Method

SpanUQ fuses multi-layer LLM hidden states, encodes tokens, then uses a DETR-style decoder to detect spans and estimate uncertainty via Mixture of Beta. Uncertainty-Conditioned Iterative Refinement (UCIR) refines these estimates.

In practice

• Deploy lightweight probes (~25M parameters) for LLM uncertainty quantification.
• Utilize DETR-style decoders for joint span detection and uncertainty scoring.
• Employ multi-sample distillation to generate robust uncertainty labels for training.

Topics

• Span-Level Uncertainty
• LLM Hallucination Detection
• DETR Architecture
• Mixture of Beta Distributions
• Uncertainty Quantification
• Natural Language Generation

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• IUQ: Interrogative Uncertainty Quantification for Long-Form Large Language Model Generation — IUQ quantifies uncertainty in long-form LLM outputs using inter-sample consistency and intra-sample faithfulness.
• Mind the Unseen Mass: Unmasking LLM Hallucinations via Soft-Hybrid Alphabet Estimation — SHADE improves LLM uncertainty quantification by adaptively fusing missing-mass extrapolation and spectral graph analysis, especially with few samples.
• Handbook of Rough Set Extensions and Uncertainty Models — Rough set theory models uncertainty by approximating concepts through various granulation and uncertainty semantics.
• The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models — A new taxonomy and evaluation framework reveal that consensus-based methods excel in quantifying LLM uncertainty, which decreases with model scale.
• A better method for identifying overconfident large language models — Cross-model disagreement combined with self-consistency more reliably identifies LLM overconfidence and hallucinations.
• OffQ: Taming Structured Outliers in LLM Quantization by Offsetting — OffQ tames LLM activation outliers for effective low-bit quantization by offsetting concentrated high-magnitude activations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.