What makes something easy (or difficult) to forecast? This question was answered by Prof. Rob Hyndman on the Forecasting Impact podcast (February 6, 2021), and it's worth a look at his response.
Rob was recently named a Fellow of the International Institute of Forecasters, and is someone who is known (or should be known) by everyone in the forecasting profession. As a young statistician in 1998, Rob co-authored (with Makridakis and Wheelwright) the classic Forecasting: Methods and Applications (3rd edition) -- the first forecasting book I owned. A few years later, as still a young researcher, he became Editor-in-Chief of the International Journal of Forecasting, which he ran from 2005-2018. On a personal note, Rob graciously contributed original pieces to both Business Forecasting: Practical Problems and Solutions (Wiley, 2015) and Business Forecasting: The Emerging Role of Artificial Intelligence and Machine Learning (Wiley, 2021).
Hyndman's 5 Conditions for Easy (or Difficult) Forecasting
In the podcast (starting at approximately 9:00), Rob defines forecasting as "the estimation of a probability distribution of a variable to be observed in the future." This, as he notes, is a very statistical definition -- which is exactly what we would expect (and want) from a statistician. [I was pleased Rob's definition is not inconsistent with my definition of a forecast as a "best guess at what is going to happen in the future." Only his definition is a lot more sophisticated!]
By either definition anything is forecastable, but not everything can be forecasted easily or accurately. That's where Rob's five conditions come into play, describing something that is easy to forecast:
- You understand the factors that contribute to its variation.
- There is a lot of data available.
- The forecasts cannot affect the thing you are trying to forecast (so there is no feedback -- your forecasts do not make a difference to the outcome of the thing you are forecasting).
- The future is similar to the past.
- There is relatively little natural or unexplainable variation.
I'd like to compare these conditions to my treatment of the topic in The Business Forecasting Deal (p.7):
...the best we can ever do is discover the underlying structure or rule guiding the behavior that is being forecast, to find a model that accurately represents the pattern of behavior, and then pray the behavior pattern doesn't change in the future.
Assume we do discover the underlying structure of the behavior, we correctly model that structure in our forecasting software, and the structure does not change in the future. Should we then be able to achieve perfect forecasts? Unfortunately the answer is no. In any complex business or social system (including things like the buying behavior of customers), there remains an element of randomness. Even though we know the underlying structure and model the behavior correctly, our forecast accuracy will still be limited by the amount of randomness, and no further improvement in accuracy will be possible.
Again, I'm happy to find general consistency between these two characterizations of the conditions for easy / accurate forecasting. Rob's # 1, 4, 5 correspond directly to points in my characterization, and his point #3 is an important addition -- something I overlooked. For example, a forecast for low demand might impact sales and marketing policy decisions, which then create an increase in demand, resulting in poor accuracy of the original forecast.
Although having a lot of data available (Rob's #2) is always a good thing, in some cases we may have other ways of knowing what drives the variation in behavior. For example, in games of chance (such as rolling dice or tossing a fair coin), we don't need data on past rolls or tosses to estimate the probability distribution of the future outcomes. But games of chance are just a tiny subset of the kinds of real-life forecasting challenges we face, and for which lots of data will be helpful.
Check out Rob's full Forecasting Impact interview for more interesting backstory on his career, and for a humorous account of how he became co-author with Makridakis and Wheelwright of the classic text.
