Re-thinking the design choices of application data quality

If we look at how most data quality initiatives start, they tend to follow a fairly common pattern: Data quality defects are observed by the business or technical community Business case for improvement is established Remedial improvements implemented Long-term monitoring and prevention recommended Move on to the next data landscape […]

Post a Comment

Sisyphus didn’t need a fitness tracker

In his pithy style, Seth Godin’s recent blog post Analytics without action said more in 32 words than most posts say in 320 words or most white papers say in 3200 words. (For those counting along, my opening sentence alone used 32 words). Godin’s blog post, in its entirety, stated: “Don’t measure […]

Post a Comment

Lack of knowledge and the root-cause myth

A lot of data quality projects kick off in the quest for root-cause discovery. Sometimes they’ll get lucky and find a coding error or some data entry ‘finger flubs’ that are the culprit. Of course, data quality tools can help a great deal in speeding up this process by automating […]

Post a Comment

Data science versus narrative psychology

My previous post explained how confirmation bias can prevent you from behaving like the natural data scientist you like to imagine you are by driving your decision making toward data that confirms your existing beliefs. This post tells the story of another cognitive bias that works against data science. Consider the following scenario: Company-wide […]

Post a Comment

Cracking the code for a successful conversion: establishing security

What kind of security do we need for this conversion?  In fact, where are the security people?  Including security personnel upfront in any conversion project can sure save some time and heartache later.  It is important to include security for the following: Source system access – You must be able […]

Post a Comment

Video tutorial: 5 ways to instantly improve your data profiling performance

Data profiling is essential. So why do so many data quality teams fail to get the most out of this crucial technique? In my short video, you’ll discover the answers to unlocking the full potential of your data profiling efforts. By broadening and deepening your knowledge of data profiling with […]

Post a Comment

Can data change an already made up mind?

Nowadays we hear a lot about how important it is that we are data-driven in our decision-making. We also hear a lot of criticism aimed at those that are driven more by intuition than data. Like most things in life, however, there’s a big difference between theory and practice. It’s […]

Post a Comment

On pronouns, online dating and data laziness

Working from home confers significant benefits. Two of my favorites are a two-second commute and the ability to take afternoon naps without offending judgmental coworkers. Among the drawbacks, though: I'm not going to randomly meet someone at the office. Like many single professionals, I have dabbled in the world of […]

Post a Comment

Bring the noise, boost the signal

Many people, myself included, occasionally complain about how noisy big data has made our world. While it is true that big data does broadcast more signal, not just more noise, we are not always able to tell the difference. Sometimes what sounds like meaningless background static is actually a big insight. Other times […]

Post a Comment

How to keep your data migration project on the rails

Why do so many data migration projects fall off the rails? I’ve been asked this question a lot and whilst there are lots of reasons, perhaps the most common is a bias towards finding the wrong kind of data quality gaps. Projects often tear off at breakneck speed, validating and cleansing […]

Post a Comment