Aggregation of Statistical Evidence under Exchangeability

2026-07-20 · Source: stat.ML updates on arXiv.org · Field: Science & Research — Mathematics & Computational Sciences, Research Methodology & Innovation, Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

The paper "Aggregation of Statistical Evidence under Exchangeability" by Schrab et al., submitted on 17 Jul 2026, introduces a novel framework for aggregating statistical evidence. This method addresses unknown and potentially complex data dependencies by leveraging group-invariance and permutation-based constructions. It treats transformed datasets as exchangeable units, aggregating evidence across statistics for each transformed dataset and then calibrating these aggregates across transformations. The authors develop a finite-sample power and adaptivity theory for this framework, extending it to sequential and data-dependent aggregation while preserving validity. For single-batch aggregation, the critical values uniformly improve upon deterministic calibrations like Bonferroni correction, adapting to unknown dependence. The work also presents a sequential alpha-spending version for early rejection and a two-batch extension to separate standardization from calibration, reducing computation and accommodating learned aggregation rules. Applications include adaptive nonparametric testing and conformal prediction.

Key takeaway

For research scientists developing robust statistical inference methods, this framework offers a significant advancement. You should consider integrating its group-invariance and permutation-based aggregation techniques to improve the validity and power of your analyses, especially when dealing with unknown or complex data dependencies. The ability to uniformly improve on methods like Bonferroni correction and adapt to dependence structures can lead to more reliable conclusions and sharper results in areas like nonparametric testing and conformal prediction.

Key insights

A framework aggregates statistical evidence under complex dependencies using permutation-based group-invariance and calibrated transformed datasets.

Principles

• Group-invariance and permutation-based constructions enable robust evidence aggregation under unknown dependencies.
• Critical values from single-batch aggregation uniformly improve on deterministic calibrations like Bonferroni correction.
• Separating standardization from calibration via a two-batch extension reduces computation.

Method

The method aggregates evidence across statistics for each transformed dataset, then calibrates these aggregates across transformations. It extends to sequential and data-dependent aggregation, including single-batch, sequential alpha-spending, and two-batch versions.

In practice

• Sharpen existing aggregation methods in adaptive nonparametric testing.
• Improve validity and power in conformal prediction applications.

Topics

• Statistical Evidence Aggregation
• Exchangeability
• Group-Invariance
• Permutation Testing
• Nonparametric Testing
• Conformal Prediction

Best for: AI Scientist, Data Scientist, Research Scientist

Related on AIssential

• CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation — CARE improves LLM-as-a-judge evaluation by explicitly modeling and separating true quality from shared latent confounders.
• Adaptive Nonparametric Perturbations of Parametric Models with Generalized Bayes — The generalized Bayes method corrects parametric model misspecification, ensuring robust and efficient inference without complex nonparametric Bayes factor computation.
• How Useful is Causal Invariance for Domain Adaptation in Finite-Sample Settings? — Causal invariance improves supervised domain adaptation in finite-sample settings when target-risk margins between candidate models are sufficiently large.
• Robust Subgroup Analysis for Heterogeneous Censored Data — RISA-ADMM robustly identifies data subgroups and estimates covariate effects in censored, heterogeneous data, even with heavy-tailed errors.
• LOTTERY: Learning from Reference-Only Samples in Two-Sample Testing under Size Asymmetry — LOTTERY enables robust two-sample testing in few-shot, imbalanced settings by learning reference-dependent representations from abundant reference data.
• External Clustering Validation by the Homogeneity-Parsimony Trade-off — A new framework quantifies the homogeneity-parsimony trade-off in external clustering validation using information-theoretic scores, unifying evaluation criteria.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.