The International Journal of Forecasting has published its 2020-Q1 issue, guest edited by Spyros Makridakis and Fotios Petropoulos, and dedicated entirely to results and commentary on the M4 Forecasting Competition. This issue should be of great interest and value to business forecasting practitioners, and you get online access to it now as a member of the International Institute of Forecasters. (See membership promotional information below.)
M4 participants forecasted 100,000 time series. The 49 competitors included eight that used either pure Machine Learning approaches, or ML in conjunction with traditional time series methods. The six pure (or combination of pure) ML methods fared poorly, with all of them falling below a benchmark combination of three simple time series methods. However, utilizing ML either in combination with statistical methods (and for selecting weightings), or in a hybrid model with exponential smoothing not only exceeded the benchmark, but performed at the top.
With all the hype about ML methods in forecasting, the M4 provides an important source of research data on the performance of various traditional and emerging forecasting approaches. We've seen in prior research and competitions (such as the M3 in 1998), simpler methods and combination methods tend to perform well. Again, in M4, combinations of traditional time series methods ranked near the top. But the two best performers used statistical + ML methods, showing that highly complex methods can potentially forecast better than simpler methods...but at a cost.
A hugely important contribution of the M4 was in replicating (most of) the competing methods on a standard hardware configuration. This allowed us to see the "cost" of each method (in terms of computational time). For the benchmark Naive1 ("no change") model, it took 12 seconds to forecast the 100,000 series. The top performing statistical + ML methods took 6 days and 32 days respectively!!!
This issue of the IJF is a must-read for anyone interested in improving their forecasting performance. It contains 35 articles, including a foreword by Nassim Nicholas Taleb (author of The Black Swan), analyses of the results, detailed descriptions of several of the competing methods, and many discussions and commentaries (including my own on "The value added by machine learning approaches in forecasting").
IIF Membership Promotion Expires January 15
Join the International Institute of Forecasters and receive both hardcopy subscriptions and online access to two journals:
- International Journal of Forecasting
- Foresight: The International Journal of Applied Forecasting
Through January 15, new IIF members receive a bonus Foresight Guidebook (choices below), plus free access to the IJF M4 issue. Current members already have online access to the IJF M4 issue, and get extra savings (of $20 or $30) on 2- or 3-year renewals.
* Foresight Guidebooks are collections of previously published articles on specific topics. Choices are:
- Artificial Intelligence
- Forecast Accuracy Measurement
- Forecasting Product and Temporal Hierarchies
- Forecasting Method Tutorials
- Forecasting Process: Guiding Principles
- Improve Your Forecasting Process
