The Stone in the Lake: What I Learned Trying to Forecast London’s NO2

2026-07-26 · Source: Machine Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Novice, medium

Summary

An analysis of daily NO2 data from London's Kensington and Chelsea area (2010-2023) details a forecasting project. The process involved extensive data cleaning, addressing negative values, 2,952 initial missing readings, duplicates, and outliers, using interpolation for gaps under three days. Decomposition revealed a significant NO2 trend decline from 2016 and strong yearly seasonality. Autocorrelation Function (ACF) analysis confirmed both yearly and subtle weekly seasonality. The project compared baseline models (Naive, Seasonal Naive, Mean, Drift) against more complex SARIMA (1,d,1)(1,1,1,7) and (2,d,2)(1,1,1,7) models, and Meta's Prophet. Evaluation using 5-fold cross-validation with 60-day test windows showed Prophet as the most stable model, maintaining errors in the 7-9.5 range, while SARIMA models were less consistent and did not always outperform the simpler Seasonal Naive baseline.

Key takeaway

For data scientists and AI students tackling time series forecasting, prioritize rigorous data cleaning and establish strong baseline models before deploying complex algorithms. Your models, even advanced ones like SARIMA, may not consistently outperform simple baselines, and their stability across diverse test periods is as crucial as their average error. Always use cross-validation to assess true performance and avoid misleading conclusions from single test sets, ensuring your predictions are reliable and robust.

Key insights

Complex time series models do not automatically outperform simpler baselines, and stability across test periods is critical.

Principles

• Time series data inherently reflects past values through autocorrelation, trend, and seasonality.
• Minor data quality issues can significantly skew model learning and predictions.
• Model stability across diverse test windows is as important as its average error rate.

Method

Clean time series data by addressing missing values and outliers, decompose into components, establish baselines, and evaluate models using cross-validation for robust comparison.

In practice

• Use `.asfreq("h")` to identify hidden data gaps.
• Flag outliers with rolling median + MAD instead of deletion.
• Interpolate short data gaps (<3 days) using `method='time'`.

Topics

• Time Series Forecasting
• NO2 Pollution
• Data Cleaning
• SARIMA
• Prophet
• Cross-validation
• Autocorrelation Function

Best for: AI Student, Data Scientist

Related on AIssential

• From Time Series Confusion to Forecasting London’s NO₂: What My First Real Forecasting Project... — Real-world time series forecasting demands robust data cleaning and appropriate validation beyond standard models.
• A Free Baseline That Beats Amazon’s Neural Net — Probabilistic forecasting should always validate complex models against naive baselines to ensure true value.
• ARIMA & SARIMA: Classic Statistical Models for Time Series — ARIMA and SARIMA models effectively forecast time series by combining autoregressive, integrated, and moving average components, with SARIMA adding seasonal capability.
• Measuring Structure Stability of Econometric Models — Model stability, distinct from accuracy, is crucial for robust econometric forecasting, especially with temporal data.
• Benchmarking Counterfactual Prediction in Epidemic Time Series with Time-Varying Interventions — A new benchmark provides realistic, observable counterfactuals for epidemic time series under dynamic interventions, addressing a critical gap.
• Why a Model with No Predictive Power Can Score 90% Accuracy — Look-ahead bias from overlapping outcome windows can inflate model accuracy significantly, even with no predictive power.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning on Medium.