Changing the paradigm for business forecasting (Part 3 of 12)

Anomalies: The Beginning of a Crisis

While even trained scientists can fail to see things that fall outside what they are looking for, anomalies eventually start to get noticed. But still, for a long time, anomalies within an existing paradigm are seen as mere “violations of expectation.” The response within the community is to figure out what went wrong – was there a problem with the measurement, or instrumentation, or was there just error in the way the observation was being interpreted?

But eventually, when there are enough observations that cannot be explained under the existing paradigm, then science is thrown into a crisis. It is a period of intense research, and the examination of alternatives.

The only way out of the crisis is the adoption of a new paradigm – a paradigm shift – a phrase that is now in the vernacular thanks to Kuhn’s book.

The Crisis in Business Forecasting

I don’t think we’re yet in this crisis state in business forecasting – although maybe we should be. And maybe readers of this blog can help provoke one.

To a large degree, the Offensive paradigm esteems the mastery of mathematics over an understanding of how business forecasting actually works.* The question is not where the Offensive paradigm has taken us, or whether it has been successful. The Offensive paradigm has been wildly successful, and taken us a very long way over the last 60 years.

The question is where to go next.

Can big data, more esoteric modeling, and more elaborate forecasting processes take us to the next level of improvement in real-life business forecasting? That is my concern. What we’ve failed to see, the anomaly that’s right in front of our noses, is that there is scant evidence this approach can continue to be counted on to substantially improve our forecasts.

High on Complexity

Paul Goodwin

Dr. Paul Goodwin

In a 2011 article** published in Foresight: The International Journal of Applied Forecasting, Paul Goodwin referred to this as a love affair with complexity. Goodwin's article pointed to several recent examples of papers…papers that are almost comical to look at.

Here are a couple of the newly proposed techniques, and the evidence offered for them:

  • Analytical Network Process – a technique based on relatively complex mathematics that allowed experts to systematically structure their knowledge of the key drivers of sales to make their judgmental forecasts more consistent and accurate.

So it is a way to assist in making judgmental forecasts.

The approach yielded forecasts with a 1.3% error, which was minimal compared to a benchmark of six common statistical techniques applied to the data. This looks great! I’ve been involved in forecasting for over 30 years and I never reach a 1.3% error.

But Goodwin pointed out that the six common techniques the method was compared to were not really appropriate for the type of data they were forecasting, so it isn’t surprising they didn’t do very well. What’s worse, the authors failed to make the most obvious and useful comparison – to judgmental forecasts unaided by their analytical network process. Further, the methods had only been tested on one sales figure! But this didn’t stop the authors from providing 33 pages of discussion and 9 tables of results – including two 13x13 matrices containing figures to five decimal places.

  • Seasonal Hybrid Procedure -- a technique that uses an adaptive particle-swarm optimization algorithm to forecast electricity demand.

The model was fit to 57 past monthly observations, and used to generate 9 months of out of sample forecasts. The authors asserted their model was “an effective forecasting technique for seasonal time series with nonlinear trend.” They asserted this without even testing it over a full 12 months!

Goodwin’s conclusion was that “If the name of the method contains more words than the number of observations that were used to test it, then it’s wise to put any plans to adopt the method on hold.” Whoever said that forecasting professors couldn’t be funny?


*This phrase was adapted from John Naughton's 2012 article "Thomas Kuhn: The man who changed the way the world looks at science," in a shrewd comment on the 2008 financial crisis: "...social scientists saw the adoption of a paradigm as a route to respectability and research funding, which in due course led to the emergence of pathological paradigms in fields such as economics, which came to esteem mastery of mathematics over an understanding of how banking actually works, with the consequences that we now have to endure."

**Paul Goodwin, "High on Complexity, Low on Evidence: Are Advanced Forecasting Methods Always as Good as they Seem?" Foresight 23 (Fall 2011), 10-12.

[See all 12 posts in the business forecasting paradigms series.]

Post a Comment

Changing the paradigm for business forecasting (Part 2 of 12)

The Current Paradigm for Business Forecasting

Image of monographSo what is the current paradigm that we, the community of business forecasting practitioners and researchers, are operating under? I’d argue that for at least the last 60 years, since 1956 when Robert G. Brown published his short monograph Exponential Smoothing for Predicting Demand, that business forecasting has been dominated by what I’ll call the “Offensive” paradigm.

I mean “offensive” in the sense of “playing offense” as in sports, not in the sense of hurting someone’s feeling. Although in the daily practice of business forecasting, feelings are bound to get hurt!

The offensive paradigm is characterized by the development of models and methods and organizational processes that are designed to extract every last fraction of a percent of accuracy in our forecasts.

More is thought to be better – more data, bigger computers, more sophisticated models – and more elaborate collaborative processes that let more people add more (of the grossly mis-named) “management intelligence” to the forecast.

Of course we no longer go to seers on mountaintops for our forecasts. This modern era of Offensive forecasting has taken us a very long way. Particularly in the ability to:

  • Automatically diagnose each time series
  • Automatically build an appropriate customized forecasting model for each time series
  • Automatically generates forecasts.

And do this quickly and on an unimaginable scale.

But in terms of progress toward more accurate forecasts, are we now stuck? Is the Offensive paradigm reaching its limits?

The Paradigm Limits What You See

Your paradigm helps you make sense of observations, and guides the kind of questions you ask and the research you do. But the paradigm can also influence the way you observe phenomena, and in fact may limit what you observe to phenomena that adhere to the paradigm.

There is a large body of literature on how we see what we want or expect to see, and can ignore things that are unexpected or don’t fit the story. Most of you are probably familiar with this famous experiment, but let’s take a look.

Gorilla Experiment Video[play video]

Did you correctly count the passes? I did. But about half the people who see this video for the first time fail to see the gorilla. I was one of them.

But even if you noticed or knew to look for the gorilla, did you notice anything else? That the curtain changed color from red to gold, and that one person in a black shirt walked off?

The gorilla experiment is a good illustration that we often don’t notice the unexpected, things we aren’t looking for, or things that fall outside our paradigm. Sometimes we don’t notice because it is things we don’t want to see – because it will rock our world in an unpleasant or inconvenient way. I suspect we all can think of situations where we’ve succumbed to this -- where we’ve failed to see what’s right in front of our nose.

One might expect that trained observers, like scientists, or professionals in some field, couldn’t make the same mistake. But that doesn’t seem to be the case. Because of the paradigm they are operating under, their view of the world and how it behaves and their expectations, even trained observers may fail to see the obvious.

In a 2013 study published in Psychological Science, radiologists were asked to examine scans of patients’ lungs and search for nodules indicative of cancer.  How many do you see? (click image to enlarge)

gorillascan

One of the scans – this one – included an image of a gorilla that is obvious when you are looking for it, yet apparently not so obvious when you are looking for something else. Although the gorilla is 48 times the size of the typical nodule, and eye tracking showed most of the radiologists looked right at it, 83% of them failed to notice.

[See all 12 posts in the business forecasting paradigms series.]

Post a Comment

Changing the paradigm for business forecasting (Part 1 of 12)

In a February 2015 post Offensive vs. Defensive Forecasting, I sought to distinguish two very different approaches to the business forecasting problem:

  • Offensive: The "offensive" forecaster is focused on forecast accuracy -- on extracting every last fraction of a percent of accuracy we can hope to achieve. The approach is to gather more data, build more complex models, utilize more elaborate forecasting processes, and incorporate more human inputs into the process.
  • Defensive: A "defensive" forecaster is not so much concerned with accuracy in itself, but in the overall effectiveness of the forecasting process. A defensive forecaster recognizes that there is a limit on the level of accuracy we can hope to achieve, and is therefore focused on achieving a level of accuracy that is reasonable to be expect (given the nature of what is being forecast) and do this efficiently (without the wastefulness of non-value adding activities that increase bias or reduce forecast accuracy).

Last week I delivered the opening keynote at the Foresight Practitioner Conference -- a unique event with a theme of "Worst Practices in Business Forecasting." This is the first of a 12-part blog series based on that keynote, in which I advocate a change from an offensive to a defensive paradigm for business forecasting.

The Structure of Scientific Revolutions

Some of the more seasoned readers of The BFD may recall an important and influential book published over 50 years ago, Thomas S. Kuhn’s The Structure of Scientific Revolutions. I was recently reminded of this book when in appeared in Parade Magazine's list of The 75 Best Books of the Past 75 years.

Book CoverFor me, as a philosophy and math student in the 1970’s, Kuhn’s book was required reading. But the book’s argument, about the nature of the progression of scientific knowledge, has had influence well beyond its original intended audience. And I think it can be applied to business forecasting today.

One word that will forever be associated with the book is paradigm. But what is a paradigm? And what did Kuhn mean by it?

You can think of a paradigm as a worldview, a framework, a natural human way to see and comprehend everything that’s around us.

We need a paradigm to manage and explain all the observations we are constantly bombarded with. The world would not make any sense if it were all just random sights and sounds and tastes and touches. We need a paradigm to organize our perceptions, and make them understandable.

Normal Science

The specific question Kuhn began considering is how does science progress?

It seems to progress like this – that over time we accumulate more observations or facts, we fine-tune our theories to explain those facts, and create experiments to test those theories. This is what Kuhn called “normal science” – a long, gradual, continuous progression of scientific knowledge.

Kuhn contends that the practitioners of normal science are guided by a paradigm, a shared worldview, a common understanding of what the world is like. The paradigm legitimatizes the problems or “puzzles” that the scientific community works on. Normal science proceeds by filling in the gaps and fleshing out the details -- all within the context of the paradigm that the community believes in.

As an historical example, for thousands of years astronomy was practiced under a geocentric paradigm, with the earth as the center of the universe. The moon, the planets, and the sun circle the earth each day. And even the distant stars revolved around the earth in a celestial sphere.

Over the centuries as more observational data was gathered, and measurement improved, minor adjustments had to be made. For example, planets sometimes appear to be moving backwards against the backdrop of the stars, a phenomenon known to as “retrograde motion.”

So the geocentric model had to be tweaked, with the notion of small “epicycles” being added to the circular planetary orbits. This provided an explanation for the anomalous observations, while still preserving the overall geocentric paradigm.

[See all 12 posts in the business forecasting paradigms series.]

Post a Comment

NIJ crime forecasting challenge

NIJ's Real-Time Forecasting Challenge

If you want to show off your forecasting chops, and maybe even make a little money, the National Institute of Justice has just the challenge for you. The NIJ's Real-Time Crime Forecasting Challenge:

...seeks to harness the advances in data science to address the challenges of crime and justice. It encourages data scientists across all scientific disciplines to foster innovation in forecasting methods. The goal is to develop algorithms that advance place-based crime forecasting through the use of data from one police jurisdiction.

A total of $1.2 million is being awarded across three contestant types:

  • Large Business Contestants: 40 prizes of $15,000 each for a total prize of $600,000.
  • Small Team/Business Contestants: 40 prizes of $10,000 each for a total prize of $400,000.
  • Student Contestants: 40 prizes of $5,000 each for a total prize of $200,000.

Full details are available on the forecast challenge website, and you can register for an information webinar on October 6, at 1pm ET.

Post a Comment

New course on Next Generation Demand Management

My friend and colleague Charlie Chase, author of the new book Next Generation Demand Management, has developed a 2-day course to go along with the book. The course is part of the SAS Business Knowledge Series, and is being offered in Chicago, October 19-20. Here are the details:

Next Generation Demand Management: People, Process, Analytics, and Technology

Presented by Charles Chase, Jr., Advisory Industry Consultant, SAS Retail and Consumer Packaged Goods Global Practice

Book coverThis course focuses on the next generation demand management with a focus on people, processes, analytics, and technology -- with an emphasis on analytics -- that drive improved demand forecasting and planning within manufacturing companies.

With this two-day course you will be able to implement a multi-step demand management process and champion the critical shift in leadership to change the corporate culture necessary to drive adoption and accountability, while never losing focus on continually improving demand forecasting and planning excellence.

Commercial and demand forecasting professionals at large consumer products, electronics, sportswear, appliance, automotive, pharmaceuticals, and other related manufacturing companies, as well as retailers, can master the key principles of demand-driven planning, gain new analytics skillsets, and implement a demand-driven planning process that your company can implement to transition to the next generation demand management.

This course is a combination of lecture, demonstrations, and hands-on exercises using the latest SAS Demand-Driven Planning solutions. Students will be able to follow along with the instructor during demos that use the actual technology.

Learn how to

  • develop the next generation demand management organization
  • use demand sensing and pattern recognition to support a demand forecasting and planning
  • define demand sensing, demand shaping, and demand shifting
  • define data requirements, skill, and technology capabilities
  • apply statistical modeling techniques, starting with moving averaging, exponential smoothing, dynamic regression, ARIMA, and ARIMAX methods
  • interpret and apply causal models
  • measure the effects of promotions (lifts)
  • evaluate the adequacy of models
  • build holistic models that account for key business drivers (demand signals) that influence demand (trend, seasonality, price, in-store merchandising, sales promotions, advertising, and more)
  • apply consumption based models using a MTCA (Multi-tiered Causal Analysis) process to measure the push/pull effects of a company's business.

Who should attend
Supply chain directors, directors of demand forecasting and planning, demand managers, forecast managers, marketing analytics managers, demand planners, forecast analysts, marketing planners, and sales planners

 

 

 

 

Post a Comment

Business Forecasting book review in JBF

Book Review in Journal of Business Forecasting

The Summer 2016 issue of Journal of Business Forecasting includes a book review of Business Forecasting: Practical Problems and Solutions. The review is by Simon Clarke, Group Director of Forecasting at The Coca-Cola Company.

You may be familiar with Clarke's many previous contributions to the forecasting literature, which include:

  • Managing the Introduction of a Structured Forecast Process: Transformational Lessons from Coca-Cola Enterprises Inc., Foresight #4 (June 2006).

and several entertaining "Joe and Simon Sez" articles co-authored with Joe Smith:

  • Who Should Own the Business Forecasting Function?, Foresight #20 (Winter 2011).
  • Forecasting Tools: Have They Upgraded the Forecasting Process?, Foresight #22 (Summer 2011).
  • Our Best Worst Forecasting Mistakes, Foresight #25 (Spring 2012).
  • Fostering Communication that Builds Trust, Foresight #28 (Winter 2013).

Attend the Foresight Practitioner Conference and Get a Free Book

Remember, if you attend the upcoming Foresight Practitioner Conference (October 5-6 in Raleigh, NC), you will receive a free copy of the new book. I will be joined by my co-editors Udo Sglavo (Sr. Director of Advanced Analytics R&D at SAS) and Len Tashman (Editor-in-Chief of Foresight) for a book signing on October 6, 7 - 8 a.m.

Book Giveaway

Post a Comment

Last day -- Foresight Practitioner Conference $200 registration discount

Companies launch initiatives to upgrade or improve their sales & operations planning and demand planning processes all the time, but many fail to deliver the results they should. Has your forecasting operation fallen short of expectations? Do you struggle with "best practices" that seem incapable of producing accurate, useful results?

Join your professional peers at the upcoming Foresight Practitioner Conference entitled "Worst Practices in Forecasting: Today's Mistakes to Tomorrow's Breakthroughs." This 1.5-day event will take place in Raleigh, North Carolina, October 5-6, 2016. Register today to take advantage of the early bird savings offer expiring on Monday, August 15. And at the event, be sure to meet the authors and receive a free signed copy of the new book, Business Forecasting: Practical Problems and Solutions.

Book Giveaway

Invited speakers will share how they and others have uncovered and eliminated bad habits and worst practices in their organizations, for potentially dramatic improvements in forecasting performance. Some of the topics to be addressed include:

  • Use and Abuse of Judgmental Overrides
  • Improper Practices in Inventory Optimization
  • Avoiding Dangers in Sales Force Input to Forecasts
  • Pitfalls in Forecast Accuracy Measurement
  • Worst Practices in S&OP and Demand Planning
  • Worst Practices in Forecasting Software Implementation

Registration will close in just over a month, and $200 in registration savings expires on Monday, August 15. Register today to reserve your seat and save!

Visit the conference web site for registration and complete conference information: https://forecasters.org/foresight/2016-conference/

Any questions? Contact:

Stacey Hilliard, Marketing Director
Foresight: The International Journal of Applied Forecasting
staceyhilliard@forecasters.org
+1 (781) 308-3334

This conference is being produced by the editorial team at Foresight: The International Journal of Applied Forecasting. Foresight is published four times a year by the non-profit International Institute of Forecasters (IIF), an unbiased, non-commercial organization, dedicated to the generation, distribution, and use of knowledge on forecasting in a wide range of fields.

Post a Comment

Next-generation demand management

Announcing New Book by Charlie Chase: Next-Generation Demand Management

Charlie ChaseMy colleague Charlie Chase has just published his latest book, Next-Generation Demand Management. It is available August 29, and can be pre-ordered now on amazon.com. It will also be available for purchase at the SAS Bookstore, along with Charlie's other books:

From the description:

Next-Generation Demand Management gives readers a better framework for building the foundation proven to fuel growth. Written by an international thought leader and practitioner in business forecasting, this next generation demand management framework radically improves the traditional supply chain demand forecasting and planning function. It moves beyond the typical demand-driven approach to a holistic view of the supply chain by identifying the commercial organization as a critical component of the unconstrained demand forecast. By doing so, it elevates demand generation, revenue, and profitability, along with customer service levels, and reduces inventory costs, waste, and working capital. Along the detailed road map, you gain the insight and tools for enhancing the skills and behaviors of your people, integrating horizontal processes, improving the accuracy of your forecasts with predictive analytics, and simplifying the entire process with scalable technology. Culled from the author’s extensive experience, this everyday guide covers only the theory directly applicable to situations encountered in the real world, which makes it completely relevant and easy to put to use right away. Take control of the big picture by:

Book cover• Identifying the most important skills the top people in demand management possess.

• Developing the internal structures and practices used by organizations leading the way in demand management.

• Fully taking advantage of big data and new technologies in order to master predictive and descriptive analytics.

Whether you’re currently in the field or plan to be soon, the revealingly illustrative examples of companies applying covered concepts in the real world enables you to implement strategies more easily the first time. Watch your revenues soar with more accurate predictions of how demand will im­pact your supply chain with Next-Generation Demand Management.

Meet Charlie at IBF Orlando

Meet Charlie (and me) at the Institute of Business Forecasting conference in Orlando, October 26-28.

Post a Comment

2016 SAS/IIF forecasting research grant

For the fourteenth year, the International Institute of Forecasters, in collaboration with SAS®, is proud to announce financial support for research on how to improve forecasting methods and business forecasting practice. The award for the 2016-2017 year will be two $5,000 grants, in Business Applications and Methodology.

Criteria for the award of the grant will include likely impact on forecasting methods and business applications. Consideration will be given to new researchers in the field and whether supplementary funding is possible.

Applications must include:

  • Description of the project (max. 4 pages)
  • Letter of support from the home institution where the researcher is based.
  • Brief (max. 4 page) c.v.
  • Budget and work-plan for the project.

The deadline for applications is September 30, 2016. 

For a complete overview of the requirements for the award, click here.

All applications or inquiries should be sent to IIF Business Director (pamstroud@forecasters.org).

The IIF-SAS grant was created in 2002 by the IIF, with financial support from the SAS Institute, in order to promote research on forecasting principles and practice. The fund provided amounts of US $10,000 per year, which is divided to support research in the two basic aspects of forecasting: development of theoretical results and new methods, and practical applications with real-world comparisons.

A list of previous grant recipients and their research is listed on the IIF website.

Post a Comment

The online SAS forecasting community

The online SAS Support Communities are a vibrant source of information and interaction for over 90,000 registered participants. Here you can ask (and answer) questions, grow (and share) your SAS expertise, and explore these collection points for other resources (like hot tips, articles, blogs, and events) relating to the community topic.

SAS Forecasting and Econometrics is one of over 30 such communities currently available on the SAS support pages.

This site should be visited and bookmarked by every user of SAS forecasting software. It contains answers on over 500 subjects, making it a great first stop when you have a coding or modeling question. (See my colleague Chris Hemedinger's informative blog about "How SAS Support Communities can expedite your tech support experience.")

Kinds of Forecasting and Econometrics Subjects

To give you a flavor for the subject matter in the community, here are some examples:

Most questions deal specifically with coding and modeling within SAS forecasting software. However, the community is also helpful for addressing broader questions on forecasting process, and for other forecasting-related announcements, such as:

Where to Begin

Lurking in your chosen community is perfectly acceptable. But I would encourage you to overcome any shyness, and to go ahead and post your questions (and answers).

Communities function best when there are lots of active participants, and the SAS communities are active. Chris Hemedinger cited recent statistics that 62% of questions get their first response within 60 minutes of posting! And 92% get responses within the first day.

Community moderators blast any unanswered questions to a corps of volunteer responders from SAS R&D, professional services, product management, and marketing. So there is a very good chance any questions you have will be answered promptly, and correctly, by an expert in the subject matter.

Post a Comment
  • About this blog

    Michael Gilliland is a longtime business forecasting practitioner and currently Product Marketing Manager for SAS Forecasting. He initiated The Business Forecasting Deal to help expose the seamy underbelly of the forecasting practice, and to provide practical solutions to its most vexing problems.

    Mike is also the author of The Business Forecasting Deal, and co-editor of Business Forecasting: Practical Problems and Solutions. He also edits the Forecasting Practice section of Foresight: The International Journal of Applied Forecasting.
  • Subscribe to this blog

    Enter your email address:

    Other subscription options

  • Archives