Managing expectations for forecast accuracy is very important, as often those expectations are extreme after management invests in a new system. Software vendors have also been known to make overly (choose one: optimistic? sanguine? idyllic?) accuracy claims as part of their sales pitch.
Of course, there is no arbitrary level of accuracy that can be promised, no matter what management wants or needs. The best way to set expectations for what is reasonable is to run a simple model (e.g. moving average) against your historical data, and see how accurate such a model would have been.
Management has every right to expect a sophisticated new statistical software package to forecast at least this well (it would have been an unfortunate investment if it couldn't!). However, the accuracy improvement due to a sophisticated model is sometimes surprisingly small -- because sometimes a simple model happens to be the most appropriate for forecasting.
Forecast accuracy is more a function of the “forecastability” of what you are forecasting than of the particular model you use to forecast it. If something is well behaved and easy to forecast, any appropriate model should do reasonably well. If something is highly erratic and difficult to forecast, no model is going to do very well, and you just have to plan your operations around that uncertainty.
Also, forecastability of a product can change over time…you may switch something from stable pricing, to making it highly promoted with frequent price changes. You can probably get more accurate forecasts when pricing is stable (and customer demand is more stable), than when you increase demand volatility by promoting it. So your accuracy expectations would change if there is such a change in how you sell/promote the product.
For setting performance expectations/objectives for the future, they can be as simple as: Do no worse than a moving average. Today I don’t know how accurate a moving average will be in 2012 (so I can’t name a specific accuracy target for next year). However, it is reasonable to expect my forecasting software and forecasting process to NOT make the forecast any worse than that.
Find Three Wrong Ways to Set Accuracy Objectives on the IBF Blog
For more discussion on this topic, including three wrong ways to set accuracy objectives, see my guest post "2012 Forecasting Performance Objectives" on the Institute of Business Forecasting blog. I want to thank IBF for the special surprise of having my favorite theoretical physicist Stephen Hawking read the blog to us by clicking on the "Listen Now" button near the top of the page. Check it out!!
