Paul Goodwin on misbehaving forecasting agents

Paul Goodwin is Professor Emeritus of Management Science at University of Bath, and one of the speakers at this fall's Foresight Practitioner Conference (October 5-6 in Raleigh, NC). His topic will be "Use and Abuse of Judgmental Overrides to Statistical Forecasts"-- an area in which he has contributed much of the important research.

Paul also contributed four articles to the recently published Business Forecasting: Practical Problems and Solutions, authoring or co-authoring:

In the Spring 2016 issue of Foresight, Paul has a new article dealing with "Misbehaving Agents."

The Principal and the Agent

An agent is a person (or organization) that is paid to act on behalf of a principal (another person or organization).

For example, you pay a real-estate agent to sell your house. But questions can arise whether the agent is acting in the best interests of the principal. (Is the agent going for a quick sale (and quick commission), minimizing the effort they have to spend on the sale? Or is the agent trying to maximize the selling price, even if it requires more time and effort?)

The Principal-Agent Problem in Forecasting

The principal-agent problem is well recognized in forecasting. Forecasting process participants (the agents being paid by the organization to develop the forecast) have a variety of interests. Those personal interests often compete with the organization's interest in receiving an honest and accurate forecast of the future. For a familiar example, consider the role of sales people as agents in the forecasting process.

In a June 26 post on SupplyChainShaman.com, industry analyst Lora Cecere stated:

Collaborative sales forecasting input leads to increased bias and error. My advice for the global supply chain leader is not waste your time asking the sales team to forecast.

I came to a similar conclusion in my Foresight article (Fall 2014), "Role of the Sales Force in Forecasting":

In the spirit of "economy of process," unless there is solid evidence that input from the sales force has improved the forecast (to a degree commensurate with the cost of engaging them), we are wasting their time -- and squandering company resources that could better be spent generating revenue.

Even assuming that sales people have special knowledge of future customer demand, there is still a question of their motivation to provide honest input to the forecast. If their sales quota is based on the forecast, it is in the sales person's interest to purposely forecast low (to get an easier-to-achieve quota). Goodwin cites a similar example from his earlier research*, where a marketing department deliberately forecasted low so they would "look good" to senior management by consistently beating it.

[Note: In his taxonomy of common forecasting misbehaviors, John Mello refers to deliberate under-forecasting as sandbagging. For the rest of his taxonomy, see "The Impact of Sales Forecast Game Playing on Supply Chains," Foresight (Spring 2009), 13-22, which also appears in Business Forecasting: Practical Problems and Solutions.]

Not a New Problem

Even back in 1957 (when future President Gerald R. Ford was still watching a lot of baseball on the radio), the problem of forecasting agents was recognized. James H. Lorie's article "Two important problems in sales forecasting" (The Journal of Business Vol. 30, No. 3 (July 1957), pp. 172-179) is so good I had to blog about it three times (Part 1, Part 2, Part 3).

Goodwin looks at other types forecasting related mischief, and reasons behind the misbehaviors. For example, new information may make it advisable to revise a forecast -- but an agent may not do so to avoid appearing wishy-washy or incompetent. Maintaining the appearance of competence is also behind unreasonably high probabilities assigned to forecasts (common among political pundits who get more air time the bolder and more confidently expressed their predictions).

Another interesting type of misbehavior, and a sure way to draw attention to yourself, is anti-herding (providing forecasts deliberately different from everyone else's). Ain't misbehavin' fun?

Solutions for Misbehaving Forecasting Agents

Metrics like Forecast Value Added can help you identify forecasting participants that are just making the forecast worse.

Goodwin suggests that forecast users (like planners and upper management) could be educated to accept the high level of uncertainty that is often unavoidable in forecasting. And it would help for reward systems to be designed to encourage honesty and accuracy (something attempted in the Gonik** system for sales force compensation developed in the 1970's for IBM Brazil).

*Goodwin, P. (1998). Enhancing Judgmental Sales Forecasting: The Role of Laboratory Research. In Wright, G. & Goodwin, P. (eds.), Forecasting with Judgement. Chichester: Wiley 91-111.

**Gonik, J. (1978). Tie Salesmen's Bonuses to Their Forecasts, Harvard Business Review, May-June 1978, 116-122.

Blogs