Many people, myself included, occasionally complain about how noisy big data has made our world. While it is true that big data does broadcast more signal, not just more noise, we are not always able to tell the difference. Sometimes what sounds like meaningless background static is actually a big insight. Other times
I sometimes refer to reference data as a “celebrity orphan” within an organization because reference data sets are touched by many business processes and applications, yet remain largely unowned and unmanaged. Few organizations have a truly formal methods for management and authority for reference data. This poses a conundrum: a
If you work in data quality long enough you’ll meet detractors of data quality software. The viewpoint in this camp is that poor quality data should be driven out at the time of design, not retrospectively detected and fixed. They perceive data quality tools as a costly overhead, something that
In my last three posts on data ethics, I explored a few of the ethical dilemmas in our data-driven world. From examining the ethical practices of free internet service providers to the problem of high-frequency trading, I’ve come to realize the depth and complexity of these issues. Anyone who's aware of these
David Loshin defines reference data and sets up a working definition for his next set of posts.
Why do so many data migration projects fall off the rails? I’ve been asked this question a lot and whilst there are lots of reasons, perhaps the most common is a bias towards finding the wrong kind of data quality gaps. Projects often tear off at breakneck speed, validating and cleansing
I have consulted on enough enterprise system implementations to know that there's anything but uniformity on how to roll out a new set of mature applications. I've seen plenty of different methodologies and technologies for relatively similar back-office systems (read: ERP and CRM). Of course, some were better than others, although
Imagine if your ability to feed your family depended upon how fast you could run. Imagine the aisles of your grocery store as lanes on a running track. If you can outrun your fellow shoppers, grab food off the shelves and race through the checkout at the finish line, then
Testing, testing, testing...do we really need testing? The answer is YES! Always! The big questions are: 1. How do I test? 2. What do I test? 3. Do we just do program testing or do we include testing data quality? 4. What about volume testing? 5. Who signs off on after-test completion?
Over the course of the last eight years, I've interviewed countless data quality leaders and learned so much about the common mistakes and failures they've witnessed in past projects. In this post I wanted to highlight five of the common issues and give some practical ideas for resolving them: #1: Not connecting
