Bayesian credible intervals are not coverage intervals — and the apologetics don’t rescue them

2026-06-04 · Source: Valeriy’s Substack · Field: Science & Research — Mathematics & Computational Sciences · Depth: Intermediate, quick

Summary

In production statistics, a persistent conflation exists between two distinct types of intervals, both commonly labeled as "90%". One interval type guarantees that the true value will be contained within its boundaries 90% of the time over a series of repeated experiments. The other, specifically a Bayesian credible interval, quantifies the posterior probability mass between its two endpoints as 0.9, derived from a given prior distribution and likelihood function. The article's title critically highlights that Bayesian credible intervals are not equivalent to coverage intervals, asserting that arguments attempting to bridge this conceptual gap are ultimately unconvincing. Understanding this fundamental difference is crucial for correct statistical inference and avoiding misinterpretation in practical applications.

Key takeaway

For data scientists and research scientists interpreting statistical results, accurately distinguishing between frequentist coverage intervals and Bayesian credible intervals is paramount. Your understanding of a "90%" interval's meaning directly impacts the validity of your conclusions and subsequent decisions. Misinterpreting a credible interval as a coverage guarantee can lead to flawed inferences about parameter ranges. Always verify the underlying statistical framework to ensure correct interpretation and communication of uncertainty.

Key insights

Bayesian credible intervals and coverage intervals are distinct, despite sharing "90%" labels, leading to frequent misinterpretation.

Principles

• Coverage intervals capture true value over repeated trials.
• Credible intervals represent posterior probability mass.
• Conflating interval types leads to statistical errors.

Topics

• Bayesian Statistics
• Credible Intervals
• Coverage Intervals
• Statistical Inference
• Frequentist Statistics
• Probability Theory

Best for: AI Scientist, Data Scientist, Research Scientist

Related on AIssential

• Bayesian credible intervals are not coverage intervals — and the gap matters — Frequentist coverage intervals and Bayesian credible intervals are distinct concepts, despite both often being labeled "90%".
• Solving a Murder Mystery Using Bayesian Inference — Bayesian Inference updates beliefs with evidence, revealing layered truths beyond initial assumptions.
• Recent discoveries on the acquisition of the highest levels of statistical fallacies — Selection bias can create spurious negative correlations among highly selected groups.
• A consensus–but it’s a consensus of uncertainty. — Consensus in statistical analysis can appropriately include agreement on uncertainty or "not knowing."
• Everywhere Valid Bounds on False Discovery Proportions in Conformal Inference — Simultaneous FDP bounds enable flexible, data-driven threshold selection with rigorous statistical guarantees.
• 5 Scipy.stats Tricks for Simulating ‘What If’ Scenarios — scipy.stats offers powerful, vectorized tools for robust probabilistic "what-if" scenario modeling, moving beyond static point estimates.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Valeriy’s Substack.