Too much information for forecasting?


First: A Report from the 67th Pine Tree Festival and Southeast Timber Expo

Back in March The BFD investigated the topic of Google-ing yourself (aka egosurfing). I reported on finding a namesake in show business, a self-described "Magic Mike Gilliland" and his sidekick Lollipop the Clown.

I attempted to disparage Magic Mike by claiming I first heard about his act in the documentary, The Aristocrats.  Apparently the decency-loving folks of Swainsboro, GA weren't paying attention, and let Magic Mike appear as planned at the 67th Pine Tree Festival and Southeast Timber Expo.1  Now, much to my further dismay, Magic Mike's act received a very favorable review on

Magic Mike Gilliland ... wowed kids of all ages with his feats of illusion mixed with healthy doses of comedy. Between shows, Gilliland and his sidekick, Lollipop the Clown, entertained festival goers with even more magic and laughs.

This was crushing news -- I always thought I was the funny Mike Gilliland.

Is this Too Much Information for Forecasting?

Consumer Goods Technology recently posted an article by Joe Shamir of ToolsGroup,  "Data, Data, Everywhere...But Most Manufacturers aren't Using It To Improve Forecasting." I generally agree with Joe's point that there are readily available sources of data we aren't taking full advantage of. (Point-of-sale data may be a prime example.) However, I did find myself disagreeing with Joe's discussion of the value of line-orders (individual customer orders for specific items).

We typically look at sales history at some level of aggregation, such as all sales of item X at location Y over some time period like week (or month).2  We utilize this aggregated history to forecast future demand for item X at location Y by week (or month).

For inventory planning purposes, forecasts are best accompanied by some indication of confidence or uncertainty. If demand patterns are fairly stable and predictable, and the forecast is for 100 +/- 10 units, this could lead to much different inventory practices than if the demand patterns are volatile and the forecast is for 100 +/- 100 units.  In the former situation we should be able to maintain high customer service (i.e. order fill rate) with less inventory than in the latter case.

I think this is the direction Joe is going when he advocates digging below the aggregate data (item / location / week level) to investigate the individual line-orders that make up the aggregate data. He argues, correctly I believe, that the statistical behavior of demand will be different, depending on the line-order makeup of the aggregate data.

For example, I would agree that if an aggregate of 48 units for an item / location / week is made up of just one order (for 48) units, the volatility of this demand stream would likely be higher than if the aggregate 48 were made up of many smaller orders. And I totally agree with the implication that demand volatility has a big impact on our ability to forecast accurately, and on how much inventory will be required to maintain service levels.

My disagreement is with the extra effort of examining the line-orders -- I don't understand why this is necessary. Why do I need to care about individual orders? Whatever volatility there is in demand will manifest itself in the aggregate (item / location / week) data!

If the line-orders are mostly single large orders and cause a more volatile demand stream, then I'll see this in the aggregate data.  If the line-orders are mostly small orders and result in less volatile demand, I'll see this in the aggregate data. I'm struggling to find the value in analyzing line-orders when I can get all the relevant information I need from the aggregate (item / location / week) data.

In short, the underlying message on the importance of demand volatility is sound. But line-orders, I believe, are TMI.


1Congratulations to Jacob Ellis, winner of the Pine Tree Festival slogan contest for his poetic "200 years of the amazing pine / have made Emanuel County fine." I haven't heard a flow like that since 50 Cent's "What Up Gansta?"  Jacob must have some Longfellow in him. Maybe he can move to Michigan and join D12 (aren't they frequently down a member?).

2Time-series forecasting models use bucketed data -- individual transactions that have been accumulated into equally spaced time buckets such as weekly or monthly. SAS forecasting software includes simple methods for accumulating transactional data into appropriate time buckets.


About Author

Mike Gilliland

Product Marketing Manager

Michael Gilliland is a longtime business forecasting practitioner and formerly a Product Marketing Manager for SAS Forecasting. He is on the Board of Directors of the International Institute of Forecasters, and is Associate Editor of their practitioner journal Foresight: The International Journal of Applied Forecasting. Mike is author of The Business Forecasting Deal (Wiley, 2010) and former editor of the free e-book Forecasting with SAS: Special Collection (SAS Press, 2020). He is principal editor of Business Forecasting: Practical Problems and Solutions (Wiley, 2015) and Business Forecasting: The Emerging Role of Artificial Intelligence and Machine Learning (Wiley, 2021). In 2017 Mike received the Institute of Business Forecasting's Lifetime Achievement Award. In 2021 his paper "FVA: A Reality Check on Forecasting Practices" was inducted into the Foresight Hall of Fame. Mike initiated The Business Forecasting Deal blog in 2009 to help expose the seamy underbelly of forecasting practice, and to provide practical solutions to its most vexing problems.


  1. Chris Hemedinger
    Chris Hemedinger on

    I think that we can all agree here that you are the funny one. It all depends on your interpretation of the word, of course.

Leave A Reply

Back to Top