I Built a Demand Forecasting Model at a Real FMCG Company. It Failed First. (Part 1)

2026-05-24 · Source: Data Science on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Retail Analytics & Intelligence · Depth: Intermediate, short

Summary

An FMCG company's data science internship involved creating a demand forecasting model to improve its Order Fill Rate, which was low due to unreliable demand planning. The initial approach used a single global CatBoost regressor on historical transaction data, including lagged demand, rolling means, product category (PH3), SKU, and sales office. This model yielded a WAPE of 44.57%, deemed unacceptable because it failed to account for diverse demand patterns across thousands of SKUs and sales offices. The revised strategy involved clustering customers first using RFM (Recency, Frequency, Monetary) and Volatility features with K-Means (K=4), then developing individual forecasting models for each segment. This improved the overall WAPE to 32.31%. However, a new issue emerged: the models forecasted demand for discontinued products, indicating a lack of product lifecycle awareness.

Key takeaway

For Data Scientists building demand forecasting models for diverse product portfolios, recognize that a single global model often fails due to varied demand patterns. You should prioritize customer segmentation using features like RFM and Volatility before developing tailored models for each group. This approach, which improved WAPE from 44.57% to 32.31% in an FMCG context, will yield more accurate predictions. Additionally, ensure your models incorporate product lifecycle information to avoid forecasting demand for discontinued items.

Key insights

Segmenting diverse demand patterns before forecasting significantly improves model accuracy.

Principles

• Diverse demand patterns require segmented modeling.
• RFM and Volatility features enable robust customer clustering.

Method

Group customers into clusters using RFM (Recency, Frequency, Monetary) and Volatility features via K-Means, then develop individual demand forecasting models for each distinct customer segment.

In practice

• Apply RFM+Volatility for customer segmentation.
• Implement separate models for distinct customer groups.
• Integrate product lifecycle data into forecasts.

Topics

• Demand Forecasting
• Customer Segmentation
• RFM Analysis
• CatBoost Regressor
• K-Means Clustering
• Order Fill Rate

Best for: Data Scientist, Machine Learning Engineer, Director of AI/ML

Related on AIssential

• The Customer Segment Nobody Wanted to Talk About. — Challenging assumptions with data can reveal overlooked customer segments with higher ROI potential than traditional VIPs.
• Predicting Risk in Content Launches: How Data-Driven Insights can Transform Launch Planning — Netflix's boosted tree models significantly enhance content launch schedule accuracy by predicting media asset delivery dates from dynamic production data.
• Integrable Elasticity via Neural Demand Potentials — ICDN precisely derives demand elasticities from learned log-demand, improving generalization and economic plausibility in multiproduct retail.
• I Built a Churn Prediction Model for a Telecom Company. Here’s What the Data Actually Revealed. — Effective churn prediction requires thorough EDA and model comparison to reveal non-linear customer behavior.
• Unified Predictive Decision Making for Retail Growth - with Felix Hoffman of 7Learnings — Unified AI-driven demand simulation across pricing, marketing, and inventory optimizes retail commercial decisions and mitigates margin loss.
• Introduction to quantitative finance Part 26: Whether to use forecasting methods, or to tell an LLM… — Forecasting in quantitative finance faces challenges with LLMs, necessitating memory-enhanced reward shaping for consistency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Data Science on Medium.