Analytics-driven forecasting means more than measuring trend and seasonality. It includes all categories of methods (e.g. exponential smoothing, dynamic regression, ARIMA, ARIMA(X), unobserved component models, and more), including artificial intelligence, but not necessarily deep learning algorithms. That said, deep learning algorithms like neural networks can also be used for demand forecasting, depending on the situation.
When it comes to demand forecasting and planning these days, many people believe demand-driven and market-driven are interchangeable. There are some basic similarities - you need to forecast true demand, and use more advanced analytics to sense and shape future demand. We all know that the true demand signal is POS (point-of-sale) channel data, and/or syndicated scanner data (Nielsen Company and Information Resources, Inc.), right? We also know that shipments are the supply signal, and sales orders are the replenishment signal. So, if you're sensing sales orders, you're not really sensing true demand, but rather replenishment. So, what signal are you forecasting?
Artificial intelligence and machine learning
There’s also been a lot of hype and discussion regarding artificial intelligence (AI) and machine learning (ML), and rightfully so given that we've solved data collection and storage challenges, as well as processing and scalability challenges. Even with all the buzz about AI and ML, not many are using either for demand forecasting - or if they are, it's on a one-off basis, not on a large-scale across the entire business product hierarchy.
In fact, I’m finding that the majority of companies who have a formal demand forecasting and planning process are still using 1990’s forecasting processes, and applying 1980's mathematical methods (e.g., moving average, and/or non-seasonal exponential smoothing). In fact, many companies feel that AI and ML is the new demand forecasting “easy button,” which I remind them only works in those Staples commercials.
You still need data scientists to monitor and tweak models using analytics-driven methods to make corrections if something dramatically changes. And demand planners are needed to own and manage the demand planning process. Close to 80 percent of a demand planner's time is spent managing data and information - not a productive use of their time. As a result, demand planners do very little real analytical forecasting.
So, why are companies so excited about using AI and ML for demand planning?
Maybe they think with all the digital disruptions, AI and ML can replace demand planners and bypass the need for data scientists, creating real-time demand execution. I’m not sure how these same companies plan to move from a 1990’s demand planning culture using 1980's mathematical methods to AI and ML overnight. Almost all of the companies we talk to are currently using moving average and/or non-seasonal exponential smoothing methods supported by Excel. They continue to cleanse the supply/replenishment history (sales orders or shipments) into baseline and promoted. We all know this is a bad practice (see "Stop cleansing your historical shipment data!")
You can’t measure all the demand patterns if your mathematical methods can only measure trend and seasonality, and then hope that collaborative “gut feeling” judgment can explain away all the unexplained variance. It requires analytics-driven forecasting and domain knowledge, not intuitive judgment.
The answer to all your supply chain challenges
Companies have been led to believe that sales & operations planning/integrated business planning (S&OP/IBP) is the “holy grail,” which will solve all their supply chain challenges. It certainly has a viable purpose, but the process is only as good as the forecast driving it. It’s the old "garbage-in-garbage-out" analogy. If you don’t believe me, then why are most companies who have implemented S&OP\IBP looking for more advanced analytical demand forecasting and planning solutions?
We've also been hearing that the forecasts being produced by legacy demand management solutions are "good enough" for companies using simple moving average\exponential smoothing methods. Even in the face of aggregate level forecasts, accuracy averages between 50 - 65 percent, and the lower mix accuracy is between 35-45 percent. So, what does “good enough” mean? Can someone tell me, please?
Companies need to move to a more analytics-oriented culture if they want to better understand their customers and more accurately predict future demand. It will take time to transition from limited analytics to a broader use because that's an enterprise effort requiring an analytics-driven corporate culture, as well as new analytics skills, horizontal processes and large scale technology.
"Good enough" forecasts driving your S&OP\IBP process is not the answer, and machine learning is not the “easy button.” You need to start with analytics-driven forecasting using large scale automatic hierarchical forecasting technology supported by data scientists who have the domain knowledge, and the advanced analytical skills to monitor, track, and tweak models as the market and consumer preferences change.
For more information, check out my book: Next Generation Demand Management: People, Process, Analytics and Technology.