I've been walking around the last few days with what looks like a dollop of chocolate syrup or grape jelly on my chin. Alas, it is just a bruise from getting elbowed in the mouth at basketball last Thursday night. (Church leagues may be the only dirtier place to play than in the SAS gym.)
After picking myself up off the ground and determining I hadn't lost any teeth, I said to the guy "Are you serious, or was that some kind of a joke?" to which he replied, "Man, I'm serious." I said, "Well that's good, because I don't take too kindly to jokes like that."
Customer Question on Naive Models
The following was forwarded from a customer of Clayton Wooddy, one of the SAS account executives:
Are companies using a specific calculation for naïve or is it just assumed that it is a basic average calculation but could vary by how many months used to create that average? Or is it specific like always just the last month’s average?
The naïve model should be something simple to calculate that can be run automatically. It serves as a forecast you can generate “for free” – what you would use if you didn’t have forecasters, forecasting software, or a forecasting process.
The two traditional naïve models (referenced in the academic literature) are the random walk and the seasonal random walk:
• The random walk (aka Naïve Forecast 1 or NF1 in the classic text Forecasting Methods and Applications by Makridakis, Wheelwright and Hyndman) just uses your last observation as your future forecast. So if you sold 12 last week, your forecast becomes 12 for all future weeks. If you sell 6 this week, you change all future forecasts to 6, etc.
• An example of a seasonal random walk (aka NF2) would be to use the actual from a year ago as the forecast for the corresponding period this year. Thus, your forecast for April 2011 would be the actual from April 2010, etc.
One thing to note is that you would never want to use the random walk as your real-life forecasting system, because your forecasts could change radically every period as the new actuals come in. (E.g. If you sold 100, 10, and 1000 units in consecutive periods, all of your future forecasts would change from 100, to 10, to 1000, creating unmanageable swings in your supply planning and operations, as well as your revenue forecasts.)
A good choice is to use simple exponential smoothing or a moving average as the naïve. Forecasts will change with new each observation, but depending on the alpha factor of your exponential smoothing (e.g. alpha = .15 is more stable than alpha = .5), or the length of the moving average (e.g. 52 week average is more stable than a 3 week moving average), the change won’t be huge and will go up or down with recent performance.
Companies have also used a composite as their naïve model – for example taking the average of a seasonal random walk (to incorporate seasonality) along with simple exponential smoothing (to incorporate the general level and trend). I think this is a great approach.
To conclude – there is no fixed or required naïve model to use. However, it should follow the principle of being simple to calculate, you should be able to automate it, and I would suggest you would also want to be able to fall back and use it as your actual forecast if your existing system and process aren’t doing any better.
Naive models can be surprisingly difficult to beat. As a critical part of FVA analysis, they are a great way to identify waste and inefficiency in your forecasting process.
3 Comments
Choosing between the random walk, the seasonal, or some other calculated version of the naive is a popular mini-controversy. The key point of the naïve is that it is simple to calculate and simpler to explain. If I found myself trying to explain how alpha works for a “naïve” exponential model, I suspect there’s a good chance that I would lose some key people in the audience, when I was supposed to be “keeping it simple.”
If we consider the naïve forecast as a “control” forecast, which is there only as a comparison for the stat forecast - we never actually forecast with it, then it’s probably okay if the naïve is a little dumb like a placebo in medical testing. In my book, the simplest “do nothing” type forecasts would be the seasonal and the random walk; both are very easy to explain.
But what happens if you use the “wrong” naïve?
Consider a product that is seasonal, but we use the random walk instead. Sure, the forecast accuracy of the naïve will be lower than it could have been, but does it still serve as a reasonable and consistent comparison for the stat model? I would say yes. Now what’s our conclusion if even though we used a not-so-good naïve, our stat model still failed to beat the naïve in 30%* of the cases? Forecast Value-added Analysis did its job; it pointed us to the areas where there is a potential for improvement. Now we can ask questions like, “Are the stat models too complicated and over-fitting the history? Are there non-repeating promotional events that should be cleaned out of the history?” and so on.
Since the goal of FVA is not to “optimize” the naïve forecast, but to drive improvement, my recommendation is to pick either the seasonal or the random walk and start using the tool to drive improvement.
* You'd be surprised how hard the naive is to beat when you start pulling real numbers!
Thanks Sean, and great remarks.
While we definitely wouldn't run our operations with a random walk model, you've pointed out the considerable merit in using a random walk (or seasonal) as the "control" forecast in FVA analysis. I buy your argument.
Looking forward to your presentation at Analytics 2011 in October.
Pingback: Editorial comment: Forecast accuracy vs. effort - The Business Forecasting Deal