Report the Floor: A Training-Free Conformal Interval Is a Mandatory Baseline for Probabilistic Time-Series Forecasting

2026-06-08 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

A new analysis demonstrates that the ConformalNaive interval, a training-free and parameter-free method, serves as a significantly stronger baseline for probabilistic time-series forecasting than currently recognized. Across 2,217 real series from nine public datasets, this simple conformal interval decisively outperforms naive value-quantile baselines, the NPTS family (beating NPTS on 73% and SeasonalNPTS on 64% of series), and the Conformal Seasonal Pools (CSP) method on 71% of series. It performs comparably to simpler learned conformal predictors like RCI and quantile regression, with median relative Winkler within 2%. While adaptive-online and ensemble methods (SPCI, ACI, AgACI) lead by 9-33% relative Winkler, ConformalNaive shows superior calibration, achieving 84-85% truth coverage at a nominal 95% compared to DeepNPTS's 66%. The study also maps a boundary where multi-step seasonal horizons favor CSP over the random-walk floor and introduces ConformalNaive+, a horizon-adaptive selector. The authors advocate for the ConformalNaive floor as a mandatory baseline for learned probabilistic forecasters.

Key takeaway

For Machine Learning Engineers evaluating new probabilistic time-series forecasters, you must integrate the ConformalNaive interval as a mandatory baseline. This training-free method provides a robust performance floor, preventing overestimation of your model's gains against weaker, traditional baselines. Ensure your evaluations cover both one-step and multi-step seasonal horizons, as optimal methods shift, and consider adaptive-online techniques for superior performance in dynamic environments.

Key insights

The simplest training-free conformal interval is a powerful, often overlooked, baseline for probabilistic time-series forecasting.

Principles

• Weak baselines distort learned forecaster performance claims.
• Conformal intervals offer robust, distribution-free uncertainty quantification.
• Performance varies significantly between one-step and multi-step seasonal horizons.

Method

ConformalNaive uses a last-value point forecast with a finite-sample split-conformal residual quantile, requiring no parameters or training. ConformalNaive+ adds horizon-adaptive selection.

In practice

• Implement ConformalNaive as a mandatory baseline for new probabilistic forecasters.
• Evaluate forecaster performance across both one-step and multi-step seasonal horizons.
• Prioritize adaptive-online methods for superior distribution shift tracking.

Topics

• Probabilistic Forecasting
• Time-Series Analysis
• Conformal Prediction
• Baseline Models
• ConformalNaive
• Uncertainty Quantification

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Conformal Prediction with Time-Series Data via Sequential Conformalized Density Regions — SCDR offers doubly robust, asymptotically valid conformal prediction for time-series, generating informative, potentially disconnected prediction sets.
• The ninety-percent confidence that means nothing — and what does — Softmax outputs are not true probabilities and misrepresent classifier confidence; conformal prediction offers reliable coverage guarantees.
• Flow-Based Conformal Predictive Distributions — Nonconformity flows enable efficient sampling of conformal prediction set boundaries in high-dimensional spaces, leading to calibrated predictive distributions.
• Rank Intervals for Leaderboards: A Hierarchical Framework for Model Evaluation — A hierarchical framework quantifies model rank uncertainty on multi-task leaderboards using statistical intervals.
• Distribution-Free Uncertainty Quantification for Continuous AI Agent Evaluation — Conformal prediction offers robust, distribution-free uncertainty quantification for continuous AI agent evaluation.
• CatBoost + Conformal: production-grade uncertainty for tabular ML — Integrating CatBoost with Conformal Prediction provides robust uncertainty quantification for tabular ML.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.