Is Complexity Bad?

It’s necessary to point out that Goodwin’s article is not arguing against complexity per se, and I’m not either.

When you have a high value forecast, where it is critical to be as accurate as possible, of course you are going to want to try every technique available to secure a good forecast. But it’s important to remember that complexity has a cost in time and effort and resources to prepare forecasts.

And while highly complex methods may impress some people, they may also be incomprehensible to managers, perhaps lessening their trust and willingness to use a forecast generated in a way they don’t understand. As Goodwin says, “It is therefore vital that recommendations to use complex methods be supported with strong evidence about their reliability.”

Simple vs. Complex Forecasting

Perhaps the most troubling problem with complex methods is that they don’t seem to forecast any better than simpler methods? Last year the Journal of Business Research published a special issue on simple versus complex methods in forecasting, led by an analysis of the evidence by Kesten Green and Scott Armstrong.

Green and Armstrong note that simplicity in forecasting seems easy to recognize, yet difficult to define. However to make practical distinctions between simple and complex forecasting, they come up with this:

Simple forecasting ≡ processes that are understandable to forecast users.

They even developed a Forecasting Simplicity Questionnaire available online so you can quiz users on whether they understand and can explain a forecasting method.

Do Simple Methods Forecast Better?

Of course the thing we really want to know is how accurate are simple forecasting methods, and do they outperform complex methods.

To find out, the authors spent two years seeking every comparative study they could find, and ended up reviewing 32 papers reporting on 97 simple versus complex comparisons. Of course, a good approach in any endeavor, forecasting or not, is to start simple and add complexity only if needed. This is just applying Occam’s Razor – do not multiply entities needlessly!

Green and Armstrong found that:

• None of the papers provide a balance of evidence that complexity improves forecast accuracy.
• Remarkably, no matter what type of forecasting method is used, complexity harms accuracy.
• ...the need for complexity has not arisen.

They state:

During our more than two years working on this special issue, we made repeated requests for experimental evidence that complexity improves forecast accuracy under some conditions... We have not been able to find such papers.

Here is a summary of their results:

Although they expected forecasts from simple methods would “generally tend to be somewhat more accurate,” the papers they studied were consistent in finding in favor of simple methods – to the extent that complex methods increased error by nearly 27% on average.

Green and Armstrong state that they found this astonishing.

[See all 12 posts in the business forecasting paradigms series.]