My previous post explained how confirmation bias can prevent you from behaving like the natural data scientist you like to imagine you are by driving your decision making toward data that confirms your existing beliefs.
This post tells the story of another cognitive bias that works against data science. Consider the following scenario:
Company-wide sales are significantly down this quarter. Ricky, the organization’s top salesperson for the last six quarters in a row, hasn't closed a single sale.
After reading that you might assume the company is doing so poorly this quarter that it’s even affecting top-performer Ricky. Now consider the following, which expands upon the same scenario with additional information:
Company-wide sales are significantly down this quarter. Ricky, the organization’s top salesperson for the last six quarters in a row, hasn't closed a single sale. Ricky recently updated his LinkedIn profile and added dozens of new connections, including several recruiters. Ricky has been using up his vacation days to take long weekend trips up state near the corporate headquarters of a major competitor whose sales are significantly up this quarter. After returning from the first of those trips, Ricky reported that his work laptop, which contained a downloaded copy of the company’s prospect database, had been stolen.
Now it seems more likely that the company is doing so poorly this quarter because Ricky, who was looking for a new job, is about to jump ship to a competitor to whom he has been giving away the company’s sales prospects.
The second version of the scenario sounds a lot more compelling, doesn't it? But why? Is it because it has more information than the first? No. It’s because the second version tells a story.
Storytelling trumps statistics
In his book You Are Now Less Dumb, David McRaney explained that the central argument of the emerging field of narrative psychology is that when attempting to understand something, we often do not use logic and careful analysis. Or construct hypotheses and test them. Or monitor and evaluate all of the variables involved.
This is because we are not natural data scientists—we are natural storytellers.
“This is narrative bias,” McRaney explained. “When given the option, you prefer to give and receive information in narrative format. You prefer tales with the structure you've come to understand as the backbone of good storytelling. Three to five acts, an opening with the main character forced to face adversity, a turning point where that character chooses to embark on an adventure in an unfamiliar world, and a journey in which the character grows as a person and eventually wins against great odds thanks to that growth. According to mythologist Joseph Campbell, that is pretty much every story every written, except for the tragedies. Those are cautionary tales in which the protagonist fails to grow, chooses poorly, submits to weakness, and as a result loses. Books, movies, games, lectures—every form of information transfer seems better when couched in the language of storytelling.”
Storytelling with statistics
My tragic tale of the turncoat Ricky has emotional appeal. The riveting account of a sales hero who betrayed the company would likely get into the heads of executive management better than any statistical analysis could.
“You have a proclivity,” McRaney explained, “for believing and accepting things more readily when they are delivered to you in story form. Raw data may be more accurate, but you'd rather simplify things and move on with your day than pore over charts and data visualizations. An emotional appeal gets into your head better than a statistical analysis. Truth and accuracy usually lose when pitted against a riveting account.”
Data science faces many antagonists, even in the most data-driven of organizations. Narrative psychology, however, might qualify as an archenemy since a data scientist must become a storyteller to present their findings. You can use narrative bias to your advantage as long as you realize that data science is storytelling with statistics.
