Monitoring data quality a little too continuously


In Nudge: Improving Decisions About Health, Wealth, and Happiness, Richard Thaler and Cass Sunstein explained how day traders with constant access to information about the value of their investments will go through a roller coaster of emotions as the market rises and falls, whereas long-term investors don’t obsessively watch the market and therefore experience less stress.

“Roughly speaking,” Thaler and Sunstein explained, “humans hate losses about twice as much as they like gains. With this in mind, consider the behavior of two investors, Vince and Rip.”

“Vince is a stock broker, and he has constant access to information about the value of all of his investments. By habit, at the end of each day, he runs a little program to calculate how much money he has made or lost that day. Being human, when Vince loses five thousand dollars in a day he is miserable - about as miserable as he is happy at the end of a day when he gains ten thousand dollars. How does Vince feel about investing in stocks? Very nervous! On a daily basis, stocks go down almost as often as they go up, so if you are feeling the pain of losses much more acutely than the pleasure of gains, you will hate investing in stocks.”

“Now compare Vince with his friend and client Rip, a scion of the old Van Winkle family. In a visit to his doctor Rip is told that he is about to follow the long-standing family tradition and will soon go to sleep for twenty years. The doctor tells him to make sure he has a comfortable bed, and suggests that Rip call his broker to make sure his asset allocation is where it should be. How will Rip feel about investing in stocks? Quite calm! Over a twenty-year period, stocks are almost certain to go up. So Rip calls Vince, tells him to put all his money in stocks, and sleeps like a baby.”

According to Thaler and Sunstein, “the lesson from the story of Vince and Rip is that attitudes toward risk depend on the frequency with which investors monitor their portfolios.”

This made me ponder the recommended best practice of continuous data quality monitoring since data quality levels also fluctuate frequently, dipping below, then rising above, the thresholds set for the acceptable level of poor data quality, which itself is analogous to investment risk.

While you can’t afford to take a Van-Winkle-like “set them and go to sleep” attitude with your data quality thresholds and hit the 20-year snooze button on your data quality monitoring, sometimes monitoring data quality a little too continuously could cause you to set off so many alarms that your data quality suffers from Chicken Little Syndrome.


About Author

Jim Harris

Blogger-in-Chief at Obsessive-Compulsive Data Quality (OCDQ)

Jim Harris is a recognized data quality thought leader with 25 years of enterprise data management industry experience. Jim is an independent consultant, speaker, and freelance writer. Jim is the Blogger-in-Chief at Obsessive-Compulsive Data Quality, an independent blog offering a vendor-neutral perspective on data quality and its related disciplines, including data governance, master data management, and business intelligence.


  1. Jim - You always come up with the greatest analogies! I use these kinds of posts to help me explain to stakeholders the nuances of data quality and data governance; it makes it easier for them to understand and appreciate (and subsequently support!).

    Thank you for all you do.

Leave A Reply

Back to Top