Changing the paradigm for business forecasting (Part 9 of 12)


Academic Research

In an approach akin to FVA analysis, Paul Goodwin and Robert Fildes published a frequently cited study of four supply chain companies and 60,000 actual forecasts.* They found that 75% of the time an analyst adjusted the statistical forecast. They were trying to figure out, like FVA does, whether the judgmental adjustments made the forecast any better.

Results chartIn the chart, they divided the adjustments into quartiles, based on the size of the adjustment. They found that large adjustments (quartiles 3 and 4) tended to improve the forecast, particularly large downward adjustments shown in red. However small adjustments had virtually impact no impact on accuracy – either positive or negative -- which when you think about it makes perfect sense.

If you make a small adjustment, even if it is directionally correct, you’ve made at best a small improvement in accuracy. So why bother?

Identifying Waste in the Forecasting Process

We see that Forecast Value Added, Theil’s U, Relative Absolute Error, and the Fildes and Goodwin study, are all going after the same thing – trying to identify waste in the forecasting process.

Activities that demonstrably fail to improve the forecast – or even make it worse – can rightfully be characterized as worst practices.

Research Agenda Under the Defensive Paradigm

Kuhn said that a scientific community’s paradigm guides how they view the world, and legitimatizes the kinds of problems the community works on. Under the Offensive paradigm, the forecasting community worked on ever more complex models and methods to extract every last bit of accuracy from our forecasts. But progress eventually stalled. It seems unlikely that some magical new modeling algorithm is going to deliver us substantially better forecasts.

So the evidence is right in front of our noses – whether we choose to see it or not – that to improve the practice of business forecasting we need to move to a new paradigm. We need a paradigm shift.

Under the Defensive paradigm, the problems and puzzles we work on will be around worst practices. This becomes our new research agenda.

Confessing Your Worst Practices

The reason for all forecasting researchers and practitioners to confess their worst practice sins is so others can learn from them. We need the business forecasting community to be exploring, trying new things, and making new mistakes – not just forever repeating the old ones.

Forecasting is a fundamental business task, yet few companies perform it efficiently and well—or at least not as efficiently and well as they would like. Managers’ attention is often misdirected to the current fads, hype, snake-oil promises – like a 2016 Gartner Supply Chain Magic Quadrant report that made it sound like “big data” was going to solve all our forecasting problems. These distractions, this strict adherence to the Offensive paradigm, can blind us to alternative ways that can truly impact forecasting effectiveness.

*Fildes, R. and Goodwin, P. (2007). Good and Bad Judgment in Forecasting: Lessons from Four Companies. Foresight 8 (Fall 2007), 5-10.

[See all 12 posts in the business forecasting paradigms series.]


About Author

Mike Gilliland

Product Marketing Manager

Michael Gilliland is a longtime business forecasting practitioner and formerly a Product Marketing Manager for SAS Forecasting. He is on the Board of Directors of the International Institute of Forecasters, and is Associate Editor of their practitioner journal Foresight: The International Journal of Applied Forecasting. Mike is author of The Business Forecasting Deal (Wiley, 2010) and former editor of the free e-book Forecasting with SAS: Special Collection (SAS Press, 2020). He is principal editor of Business Forecasting: Practical Problems and Solutions (Wiley, 2015) and Business Forecasting: The Emerging Role of Artificial Intelligence and Machine Learning (Wiley, 2021). In 2017 Mike received the Institute of Business Forecasting's Lifetime Achievement Award. In 2021 his paper "FVA: A Reality Check on Forecasting Practices" was inducted into the Foresight Hall of Fame. Mike initiated The Business Forecasting Deal blog in 2009 to help expose the seamy underbelly of forecasting practice, and to provide practical solutions to its most vexing problems.

Comments are closed.

Back to Top