Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language

2026-07-13 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

A study evaluated nine Large Language Models (LLMs) for their ability to reliably communicate probabilistic information, specifically likelihood and uncertainty, in natural language for risk communication. Models including DeepSeekR1-7B, Llama3-8B, and GPT-5.4 were tested in a two-stage pipeline, selecting verbal descriptors for simulated probabilistic outputs from Beta distributions across six domain contexts and ten temperature settings. Findings indicate LLMs are generally consistent but miscalibrated, with significantly weaker performance on uncertainty tasks. While providing precomputed summary statistics (mode and prior sample size) reduced sensitivity to contextual framing, it did not resolve the underlying miscalibration. GPT-5.4 achieved near-perfect likelihood calibration (0.96) and consistency (1.0) but scored only 0.37 for uncertainty. The research concludes that current LLMs are not yet reliable zero-shot standalone tools for probabilistic risk communication, with the primary bottleneck residing in the verbalization step.

Key takeaway

For Machine Learning Engineers deploying LLMs in high-stakes domains requiring probabilistic risk communication, understand that current models are consistent but miscalibrated, particularly for uncertainty. Do not rely on LLMs as zero-shot standalone tools for explaining likelihood and uncertainty. Instead, integrate targeted interventions at the verbalization stage or implement robust human-in-the-loop validation to ensure accurate and reliable communication of predictive risks.

Key insights

LLMs consistently misinterpret probabilistic information, particularly uncertainty, when generating natural language explanations, indicating a verbalization bottleneck.

Principles

• LLMs lack robust internal mappings for likelihood and uncertainty.
• Temperature trades off explanation stability for expressiveness.
• Miscalibration stems from verbalization, not statistical inference.

Method

LLMs selected NLEs for likelihood (mode "m") and uncertainty (prior sample size "k") from Beta distributions, then evaluated for consistency (identical outputs for identical inputs) and calibration (descriptor accuracy).

In practice

• Explicitly provide numerical summaries to reduce contextual framing sensitivity.
• Anticipate LLMs defaulting to cautious, noncommittal language.
• Target interventions at the verbalization stage for risk communication.

Topics

• Large Language Models
• Risk Communication
• Probabilistic Prediction
• Model Calibration
• Natural Language Explanations
• Uncertainty Quantification

Code references

• CTruAI/risk_communication_llm

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Ethicist

Related on AIssential

• Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs — Prompting LLMs to reason in English improves multilingual uncertainty estimation, especially for low-resource languages.
• Sound Probabilistic Safety Bounds for Large Language Models — A new framework computes rigorous, sound lower bounds for LLM harmfulness by actively guiding generation towards unsafe outputs.
• UnpredictaBench: A Benchmark for Evaluating Distributional Randomness in LLMs — LLMs struggle significantly with generating samples calibrated to true underlying distributions, hindering their use in simulations.
• Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypotheses — VPP applies Bayesian inference to natural language hypotheses, improving VML's consistency and interpretability.
• A better method for identifying overconfident large language models — Cross-model disagreement combined with self-consistency more reliably identifies LLM overconfidence and hallucinations.
• Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models — LLMs exhibit universal, scale-dependent chaotic behaviors driven by floating-point precision and error propagation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.