Love can make a person do bad, dangerous, stupid, and irresponsible things. Love of country can make a politician stray from his wife. Love of Pepsi can make a pop musician lose his hair in a pyrotechnics-gone-bad commercial. Love of acting can make academy award winners accept starring roles in Ishtar. And for me last Wednesday morning, my love of forecasting was so overwhelming and distracting, I forgot to put on shoes and arrived at work in my bedroom slippers.
Beware the "Inside View"
The November 2011 McKinsey Quarterly contains an excerpt from Daniel Kahneman's new book Thinking, Fast and Slow. Kahneman tells the story of textbook writing project, in which his survey of the co-authors estimated two years (+/- six months) for completion.
Kahneman then asked one of the co-authors if he could think of similar projects, and how long they took to completion. Here the answer was quite different -- 40% of them failed to finish, and the successful ones had taken seven to 10 years to complete.
At this realization, the more rational course of action would have been to quit. "None of us was willing to invest six more years in a project with a 40 percent chance of failure." Yet the project continued and the book was completed in eight years -- but was never used. By then the enthusiasm for the idea had waned.
Kahneman uses this story to illustrate the perils of the "inside view" -- simply extrapolating from our specific circumstances and the information in front of us. This narrow focus on what we know fails to allow for the "unknown unknowns" -- all the crises, unanticipated, and random events that get in the way of our progress. "There are many ways for any plan to fail, and although most of them are too improbable to be anticipated, the likelihood that something will go wrong in a big project is high."
The "outside view" is based on the category or reference class relevant to our prediction. Here, it was the class of similar book projects. Kahneman argues that this outside view provides "...a reasonable basis for a baseline prediction: the prediction you make about a case if you know nothing except the category to which it belongs. This should be the anchor for further adjustments." This estimate can then be adjusted given any case-specific information.
New Product Forecasting by Analogy
Knowingly or not, we employ the outside view when we forecast a new product based on sales of similar products. Such "forecasting by analogy" is perhaps the second most common new product forecasting method (second only to "management forecasting whatever they darn well please"). As Kahneman asserts, "...if the reference class is properly chosen, the outside view will give an indication of where the ballpark is. It may suggest, as it did in our [the book project]case, that the inside-view forecasts are not even close."
Selecting an appropriate reference class is the key step in forecasting by analogy, and is often poorly done. Through mischief or incompetence, we are wont to cherry pick the analogous products (e.g., only the most successful ones) in order to justify the new product forecast we desire.
Find more thorough discussion of this topic in the whitepaper "New Product Forecasting Using Structured Analogies" (available for free download).