Combining Statistical Analysis with Subjective Judgment (continued)

After summarily dismissing regression analysis and correlation analysis as panaceas for the business forecasting problem, Lorie turns next to "salesmen's forecasts."* He first echoes the assumption that we still hear today:

This technique of sales forecasting has much to commend it. It is based upon a systematic collection and analysis of the opinions of men who, among all the company's employees, are in closest contact with dealers and ultimate consumers.

But Lorie points out the "inherent deficiencies" of relying solely on sales force input, "for which it may be impossible to devise effective remedies." These are:

• Unreasonably assumes that sales people have the "breadth and depth of understanding" of the pervasive influences on demand. (Do they have any skill at forecasting?)
• Sales jobs turn over frequently, so sales people providing forecasts are often inexperienced, and we don't have enough data to determine their biases. (Will they give us an honest answer?)
• Does not incorporate "competent statistical analysis" of historical sales data which could be combined with the sales force inputs.

Lorie also disses the use of consumer surveys as costly, impractical, and unproven to be of value except in limited circumstances.

Two Solutions

The message is not all negative. Lorie provides two solutions to combing statistics with judgment, the filter technique and the skeptic's technique. I'm not as much interested in the specific techniques as in his overall approach to the problem -- which in the filter technique is to focus on economy of process. Start "with an extremely simple and cheap process to which additional time and money are devoted only up to the point at which the process becomes satisfactory."

...the process provides an objective record of both sales forecasts and the methods by which they are made so that study of this record can be a means for continual improvement in the forecasting process.

(You can find details about the filter technique in the article.)

The skeptic's technique applies process control ideas, akin to Joseph & Finney's "Using Process Behaviour Charts to Improve Forecasting and Decision-Making" (Foresight 31 (Fall 2013), pp. 41-48). Starting with "limited faith" in the persistence of historical forces that affect sales:

• Project future sales with a simple trend line.
• Compute two standard deviations on each side of the line to create a range within which future sales should fall the vast majority of time (if historical forces continue to work in the same way).

Lorie points out that this work could be done by statistical clerks "whose rate of pay is substantially less than that of barbers or plumbers."

• The forecaster then solicits company experts (who, "incidentally, usually receive substantially more than barbers or even plumbers").
• If the expert's forecasts falls within the range limits of the statistical forecast, it is accepted. If outside the limits, even after reconsideration (asking "the gods for another omen"), the forecaster has to make a decision what to do.

Lorie wryly points out that making a decision is something the forecaster has avoided up to this point.

For expert forecasts outside the statistical forecast limits, Lorie states:

...experience has indicated that the forecast in a vast majority of cases would have been more accurate if the experts' forecast had arbitrarily been moved to the nearest control limit provided by the statistical clerk rather than being accepted as it was.

In Part 3 we'll look at Lorie's remarks on the evaluation of forecasts -- and his 1957 precursor to what we now call FVA!

---------------

*The role of the sales force in forecasting is subject of my recent Foresight article (Fall 2014), and a forthcoming presentation at the International Symposium on Forecasting (Riverside, CA, June 24-27).