Recent discoveries on the acquisition of the highest levels of statistical fallacies

2026-05-13 · Source: Statistical Modeling, Causal Inference, and Social Science · Field: Science & Research — Mathematics & Computational Sciences, Research Methodology & Innovation · Depth: Intermediate, short

Summary

Alex Dimakis critiques a recent Science paper by Gullich et al., "Recent discoveries on the acquisition of the highest levels of human performance," which claims a negative association between early and peak performance among top achievers. Dimakis argues the paper makes classical statistical errors, specifically the base-rate fallacy and Berkson's paradox. He explains that while elite young performers are 40 times more likely to become top adults than the general population, the observed negative correlation among *only* top performers is an artifact of selection bias, not a universal principle. The critique highlights how such fallacies can mislead even smart people, drawing parallels to other common statistical misinterpretations, such as the incorrect assertion that causation implies correlation.

Key takeaway

For AI Scientists interpreting performance data, recognize that observed correlations within highly selected groups, like top performers, can be statistical artifacts. Do not extrapolate negative correlations between early and peak performance as universal principles without accounting for selection bias and base-rate effects. Your models and conclusions must rigorously differentiate between true causal relationships and paradoxes arising from data sampling.

Key insights

Selection bias can create spurious negative correlations among highly selected groups.

Principles

• Beware of Berkson's paradox in selected populations.
• Base-rate fallacies distort probability assessments.

In practice

• Examine selection criteria in performance studies.
• Consider regression-to-the-mean in longitudinal data.

Topics

• Statistical Fallacies
• Berkson's Paradox
• Base-Rate Fallacy
• Regression to the Mean
• Human Performance Acquisition

Best for: AI Scientist, Data Scientist, Research Scientist

Related on AIssential

• Claude Can’t Run Regression. A 200-Year-Old Theorem Proves It Never Will. — Regression estimates conditional expectations, requiring human judgment for true inference beyond pattern matching.
• The #1 model on the leaderboard dropped to #14 when I included the benchmarks they didn't report. — Selective benchmark reporting distorts AI model leaderboards, requiring comprehensive evaluation for accurate performance assessment.
• Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision — Directional confusions and Rate-Distortion geometry reveal distinct inductive biases in human and machine vision.
• Survey Statistics: Individualism and the CV Noise Problem — Individual-level log loss often fails to differentiate substantively important model improvements in binary data.
• The Boosting Paradox — Machine learning struggles in finance due to market adaptivity, adversarial competition, and irreducible uncertainty, despite theoretical power.
• Correlation is Not Causation — Correlation indicates co-movement between variables, but does not imply causation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Statistical Modeling, Causal Inference, and Social Science.