Big data and omission neglect


In my previous post, I used the book Mastermind: How to Think Like Sherlock Holmes by Maria Konnikova to explain how additional information can make us overconfident even when it doesn’t add to our knowledge in a significant way. Knowing this can help us determine how much data our decisions need to be driven by.

Another important concept Konnikova described is what is known as omission neglect.

“We fail to note what we do not perceive up front,” Konnikova explained, “and we fail to inquire further or to take the missing pieces into account as we make our decision. Some information is always available, but some is always silent—and it will remain silent unless we actively stir it up.” This is why noise is sometimes necessary.

Konnikova cited a recent study of online behavior that showed we are profoundly influenced by our personal preferences when evaluating what websites to use to find information. We use our preferences as an anchor to reduce the number of websites to consider, often returning to already known websites instead of taking the time to evaluate potential new sources of information. Even though searching the web exemplifies how the era of big data puts more information just a few key strokes, scrolls, and clicks away, we often neglect this information.

“As soon as we find an answer,” Konnikova explained, “we stop looking, whether or not the answer is ideal or even remotely accurate.” This bias is “especially strong when a plausible answer is presented early on in the search process. We tend to consider our task complete, even if it’s far from being so.” For example, even though a web search returns millions of results, how often do you scroll past the first page or click on more than a few results?

Too much, especially superfluous, information can distort decision making. As can too little information. We hear a lot about information overload, but omission neglect may be the silent killer of sound decisions.


About Author

Jim Harris

Blogger-in-Chief at Obsessive-Compulsive Data Quality (OCDQ)

Jim Harris is a recognized data quality thought leader with 25 years of enterprise data management industry experience. Jim is an independent consultant, speaker, and freelance writer. Jim is the Blogger-in-Chief at Obsessive-Compulsive Data Quality, an independent blog offering a vendor-neutral perspective on data quality and its related disciplines, including data governance, master data management, and business intelligence.

Leave A Reply

Back to Top