Everyone is familiar with The Confidence Man. This person can spout any sort of nonsense and easily demonstrable lies, and yet 40% of the people still believe him. The Confidence Man exposes a curious characteristic about human behavior: If you make your assertions with extreme confidence, no matter how obviously false or absurd, or how incoherently you assert them, there are still 40% of people who will follow you along.
I observed this myself many years ago, falling under the spell of a youngish consulting director as he swatted away hardball questions, no matter how tough or technical, from a seasoned IT management team.
I was in awe. How does a guy this young know so much?
Later, speaking privately to a colleague who knew the consultant well, I was let in on a little secret. His friend the consultant, my colleague confided, "is the most convincing BS artist he's ever heard."
So that was it! By projecting a strong, confident voice, with no hesitation in his answers and no hint of self-doubt, I (and the full management team) were believing everything the consultant said. We were not stupid people, yet we'd fallen under the spell of The Confidence Man.
The Confidence Man
First given a name in Melville's 1857 novel,* The Confidence Man has existed throughout history.
From the big city bullies and grifters we've all encountered, to the fictional characters in books and movies (Elmer Gantry, "Lonesome" Rhodes), to advisors of the powerful (Rasputin) and wealthy (Bernie Madoff), to televangelists convicted (Jim Bakker) or shamed (Jimmy Swaggart, Jerry Falwell Jr., ad nauseam), we've come to know them with great familiarity. And just like the Wizard of Oz, they are easily exposed once you pull back the curtain.
The Forecaster as The Confidence Man
Forecasters are put in the odd position of having to express confidence in something (a point forecast) of which they may have little confidence. A competent forecaster knows there is uncertainty in any assertion about the future. And yet openly expressing that uncertainty will make a forecaster look weak and waffling, and possibly lead to unemployment. So this is the inherent conflict that makes a forecaster's job such a challenge.
Management wants to feel in control of their organization -- that they alone pull the levers and they alone control the organization's fate. So expressing uncertainty in a forecast, even when that uncertainty is warranted, will not be well-received. Thus, the most common way to express the uncertainty is to not express it -- to simply provide the point forecast.
The prediction interval (PI) is the simplest way to communicate uncertainty. A forecaster might assert "The forecast is 100 +/- 10 units," with an understanding that there is a 95% chance the actual unit sales will fall between 90 and 110. Such a tight PI might well be acceptable to management. Knowing that sales are very likely to be very near 100 allows plans to be made, and actions taken.
[Unfortunately, most calculated PIs woefully underestimate the true uncertainty in the forecast, and you can make a lot of money betting that more than 1 out of 20 future actual values will fall outside the 95% PI. But this is an entirely separate matter we don't need to discuss here.]
In a more commonplace example, what does the forecaster do when the calculated PI is 100 +/- 100 units? Even if correctly calibrated, an extremely wide PI may be deemed useless for decision making. The forecaster is admitting to having little idea what is going happen next. And while that may be the only right answer, the forecaster will get laughed out of the room.
More to Come...
We'll return to this topic next time.
*The Confidence Man: His Masquerade. Does anyone else find Melville's prose insufferable?
